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FOREWORD 


The  cost  of  training  devices  and  simulators  is  not  a  trivial 
factor  in  training.  The  capability  for  simulating  reality  is 
also  increasing  on  an  annual  basis.  The  military  must  determine 
exactly  how  much  simulation  is  sufficient  for  the  learning  objec¬ 
tives.  Behavioral  and  analytical  techniques  that  can  quickly  and 
easily  project  or  predict  simulation  and  training  device  require¬ 
ments  are  lacking.  In  addition,  information  on  the  cost- 
effective  use  of  training  equipment  within  courses  of  instruction 
is  sparse.  The  development  of  models,  databases,  and  techniques 
addressing  these  problems  is  the  first  step  toward  providing 
integrated  behavioral  and  engineering  decisions  in  designing, 
fielding,  and  using  advanced  training  technology.  The  potential 
effect  on  the  Army  is  to  reduce  the  cost  of  training  equipment 
while  increasing  the  equipment's  instructional  effectiveness. 

In  order  to  address  these  concerns  and  problems,  the  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
and  the  Project  Manager  for  Training  Devices  (PM  TRADE)  have 
joined  efforts  (MOU  of  Technical  Coordination,  May  83;  MOU  Estab¬ 
lishing  the  ARI  Orlando  Field  Unit,  Mar  85;  Expanded  MOU,  July, 
86).  Task  number  3104,  Advanced  Technology  for  the  Design  of 
Training  Devices  and  Simulators  provided  the  framework  for  the 
work  reported  in  this  document.  PM  TRADE  has  been  completely 
informed  of  the  development  of  the  models  and  analytical  tech¬ 
niques.  The  models  and  techniques  developed  in  this  effort  are 
expected  to  provide  the  basis  for  useful  aids  supporting  the 
integration  of  behavioral  and  engineering  data,  knowledge,  and 
expertise  in  training  equipment  design  in  the  future. 


Technical  Director 
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SPECIFYING  SKILL-BASED  TRAINING  STRATEGIES  AND  DEVICES:  A  MODEL 
DESCRIPTION 


EXECUTIVE  SUMMARY 


Requirement: 

The  overall  Army  need  is  to  develop  models  that  can  help 
specify  effective  and  efficient  training  devices  and  strategies 
so  that  training  requirements  are  met  at  the  lowest  possible 
cost,  or  the  highest  overall  performance  may  be  obtained  at  a 
specified  cost.  Previous  efforts  have  examined  this  need  from  a 
task-based  orientation.  There  was  a  need  to  describe  model  pro¬ 
cedures  that  could  identify  the  skills  required  for  competent  job 
performance,  specify  strategies  for  training  these  skills,  design 
training  devices  to  implement  the  skill-based  training  strate¬ 
gies,  and  evaluate  the  most  efficient  use  of  skill-based  training 
devices. 


Procedure: 

The  approach  consisted  of  three  major  tasks.  The  first  task 
specified  the  requirements  of  the  model,  its  users,  and  its  major 
components  in  order  to  set  the  boundaries  of  the  approach.  The 
second  step  reviewed  the  research  literature,  addressing  the 
definition  of  skills,  the  organization  of  skill  categories,  and 
the  effectiveness  of  skill-training  strategies.  The  third  step 
developed  a  formal  model  for  specifying  skill-based  training 
strategies  and  devices,  based  on  the  knowledge  and  theory  identi¬ 
fied  in  the  first  two  steps. 


Findings: 

The  model  framework  developed  in  the  project  identifies 
three  benefits  of  skill  training.  First,  skill  training  can 
provide  much  more  practice  on  a  critical  skill  in  a  given  amount 
of  time  or  for  a  given  cost.  Second,  critical  skills  can  gener¬ 
alize  to  many  tasks,  so  that  training  the  skill  can  avoid  unnec¬ 
essary  and  redundant  task  training.  Third,  training  the  critical 
skills  involved  in  complex  and  difficult  tasks  can  decrease  the 
mental  workload  required  to  perform  the  tasks,  and  consequently 
speed  up  subsequent  task  learning. 


The  formal  model  breaks  the  specification  of  training 
strategies  and  devices  into  four  steps:  identifying  skills, 
selecting  instructional  strategies,  designing  devices,  and  allo¬ 
cating  training.  The  primary  objective  of  the  first  step  is  to 
decompose  tasks  into  their  elements  (i.e.,  performing  actions 
upon  objects) ,  to  identify  the  general  abilities  that  enable  the 
performer  to  accomplish  the  task,  and  to  describe  the  domain- 
specific  skills  that  will  become  candidates  for  skill-based 
training.  The  main  goal  of  the  second  step  is  to  group  and  se¬ 
quence  skills  for  training.  In  the  third  step,  device  require¬ 
ments  are  derived  from  the  nature  of  the  skills  to  be  taught. 
These  requirements  include  instructional  capabilities  and  inter¬ 
face  features.  The  final  activity  compares  the  projected  cost  of 
task  training  to  that  of  task  training  supplemented  by  the  pro¬ 
posed  skill  training.  A  concrete  example  from  the  Air  Traffic 
Control  domain  illustrates  the  steps  derived  from  the  theoretical 
approach  in  the  context  of  a  real  job  domain. 


Use  of  Findings: 

The  procedures  developed  in  this  effort  can  be  used  to  help 
training  developers  understand  the  benefits  of  skill  training, 
identify  effective  skill-training  strategies,  and  plan  the  effi¬ 
cient  implementation  of  these  strategies  using  training  devices. 
The  described  model  can  form  the  basis  of  an  automated  aid  for 
the  design  of  ski] 1-based  training  devices.  Finally,  the  model 
development  identified  several  needs  for  future  basic  and  applied 
research. 
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SPECIFYING  SKILL-BASED  TRAINING  STRATEGIES  AND  DEVICES: 

A  MODEL  DESCRIPTION 

INTRODUCTION 


This  report  describes  an  effort  to  develop  model  procedures  to  identify  the 
skills  required  for  competent  job  performance,  to  specify  strategies  for  training 
these  skills,  to  design  training  devices  to  implement  the  skill-training  strategies, 
and  to  evaluate  the  most  efficient  use  of  skill-based  training  devices.  A  skill-based 
training  device  may  tutor,  instruct,  or  present  new  material  to  the  student 
regarding  critical  skills  with  little  or  no  representation  of  the  job  domain, 
missions,  systems,  or  environments.  Examples  of  skills  that  might  be  trained  in 
such  devices  are  reading  radar  symbols,  identifying  vehicles,  applying  tactical 
decision  rules,  and  using  test  equipment  for  troubleshooting.  These  skills  apply  to 
many  kinds  of  tasks;  they  are  not  tasks  themselves.  The  overriding  goal  of  these 
methods  is  to  specify  training  strategies  and  devices  that  make  training  more 
effective  and  efficient,  so  that  training  requirements  are  met  at  the  lowest  possible 
cost,  or  the  highest  overall  performance  may  be  obtained  at  a  specified  cost. 

This  effort  consisted  of  three  major  tasks.  The  first  task  specified  the 
requirements  of  the  model,  its  users,  and  its  major  components.  The  second  task 
reviewed  the  research  literature  addressing  the  definition  of  skills,  the 
organization  of  skill  categories,  and  the  effectiveness  of  skill-training  strategies. 
The  third  task  developed  a  detailed  model  of  the  components  identified  in  the  first 
task.  The  results  and  conclusions  of  the  review  are  presented  in  the  second 
section  of  the  report.  The  third  section  of  the  report  presents  a  rationale  for  skill- 
based  training.  The  fourth  section  presents  an  example  that  describes  the  model 
in  the  context  of  tasks  and  skills  drawn  from  air  traffic  control.  A  detailed 
description  of  the  developed  model  is  presented  using  IDEFO  (Integrated 
Computer  Aided  Manufacturing  Definition)  methodology. 

Background 

Skilled  operation  and  maintenance  of  military  equipment  requires 
considerable  training.  The  Army  spends  approximately  $2  billion  annually  so  that 
soldiers  may  acquire  and  maintain  the  specialized  skills  (including  flight  skills) 
required  to  conduct  the  Army  mission  (Office  of  the  Assistant  Secretary  of  Defense 
(Force  Management  and  Personnel),  1989).  Because  of  the  magnitude  of  Army 
training  activities,  any  procedure  that  can  meet  training  requirements  more 
efficiently  can  have  a  substantial  impact  on  training  cost. 

Training  designers  can  enhance  training  efficiency  by  appropriate 
application  of  training  conditions,  content,  methods,  and  standards.  In 
determining  the  training  conditions,  the  designer  can  assign  the  training  setting, 
and  the  kinds  and  sequencing  of  training  equipment,  among  other  items. 
Determining  training  content  requires  the  training  designer  to  decide  what  skills 
and  tasks  should  be  the  focus  of  training.  The  training  method  can  affect  both  the 
effectiveness  and  efficiency  of  training.  Finally,  training  standards  can  affect 
retention  as  well  as  acquisition. 
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Effective  training  design  requires  a  thorough  analysis  of  job  functions  and 
activities.  At  least  two  kinds  of  analyses  are  required.  The  first  describes  the  job 
from  a  functional  viewpoint,  specifying  the  missions,  functions,  and  tasks  that 
must  be  performed  by  the  competent  job  incumbent.  The  second  determines  the 
specific  cognitive,  perceptual,  and  motor  activities  that  need  to  be  carried  out  to 
satisfy  mission  requirements.  The  analysis  of  activities  specifies  the  general 
abilities  and  specific  skills  that  are  required  for  successful  mission  performance. 

Tasks  tell  the  training  designer  what  must  be  done  to  satisfy  the  mission  of 
the  job  being  trained.  Tasks  specify  the  requirements  for  training  by  describing 
what  the  skilled  operator  or  maintainer  must  be  able  to  accomplish  with  his  or  her 
activities.  Thus,  the  performance  standards  for  training  may  be  derived  from  task 
definitions.  Task  definitions  provide  the  contingencies  for  measuring  performance, 
because  they  explicitly  link  the  outcome  of  an  activity  to  mission  performance. 

Skills  describe  the  specific  activities  required  to  perform  tasks.  By 
examining  the  general  categories  of  skills  required  to  accomplish  a  task,  the 
training  designer  can  get  a  better  understanding  of  the  perceptual,  cognitive,  or 
motor  processes  involved  in  the  task.  This  understanding  can  provide  training 
methods  that  specifically  address  the  training  needs  of  these  processes.  The 
training  designer  may  also  use  information  about  skills  to  identify  the  specific 
operations  that  are  required  for  competent  performance.  Because  skills  describe 
the  specific  processes  required  to  accomplish  a  task,  they  are  central  in  the 
definition  of  the  content  of  training. 

Task  Training  and  Skill  Training 

The  term  skill  refers  to  general,  learned  abilities  that  can  be  adapted  to  a 
particular  job  by  being  combined  with  job-specific  knowledge.  For  our  purposes, 
the  critical  attributes  of  general  abilities  are  that  they  can  be  adapted  to  any 
given  domain,  they  can  be  sharpened  with  practice,  and  they  characteristically 
distinguish  expert  from  novice  behavior.  Job  skills,  those  identified  with  a 
particular  domain,  are  viewed  as  specific  instances  of  general  abilities  applied  in 
conjunction  with  domain-specific  knowledge.  Skills  are  distinguished  from  tasks 
in  that  a  skill  is  an  ability  to  perform  some  aspect  of  one  or  more  tasks,  skills  may 
be  applied  to  several  tasks,  and  skills  are  concerned  with  processing  activities 
while  tasks  are  concerned  with  goals  and  functions. 

The  most  straightforward  approach  to  training  is  to  train  the  task 
requirements  directly.  This  method  of  training  provides  soldiers  practice  on  tasks 
in  an  environment  that  is  as  similar  to  actual  wartime  as  possible.  Training  using 
this  approach  typically  involves  the  use  of  real  equipment  or  high-fidelity 
simulators  and  realistic  training  conditions.  The  major  advantage  of  this 
approach  to  training  design  is  that  is  almost  guarantees  that  training  will  be 
relevant  to  the  requirements  (since  the  requirements  are  being  trained  directly). 
Similarly,  it  is  an  easy  matter  to  determine  training  requirements,  since  they  are 
derived  directly  from  task  performance  standards. 
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However,  the  nature  of  the  tasks  and  the  skills  required  to  perform  them 
may  interact  to  reduce  the  effectiveness  of  task  training.  Under  certain 
conditions,  some  of  the  following  disadvantages  of  task  training  may  occur. 

1.  Feedback  may  be  inappropriate.  In  the  job  environment,  the  link  between 
correct  performance  and  mission  outcome  may  be  quite  complex,  and  proper 
actions  may  not  always  produce  the  best  outcome.  For  example,  the  proper 
action  may  lead  to  a  bad  outcome  because:  (a)  there  may  not  have  been  any 
chance  of  obtaining  a  good  outcome,  (b)  there  may  have  been  some  error  in 
a  subsequent  action,  or  (c)  unforseen  or  random  events  may  have  produced 
the  outcome. 

2.  Feedback  may  be  delayed.  Feedback  is  often  delayed  in  tasks  that  involve 
planning.  For  example,  when  an  air  traffic  controller  directs  aircraft  to 
certain  heading,  it  takes  time  for  the  aircraft  to  execute  the  turn.  The 
controller  doesn’t  know  if  he  has  given  the  correct  directions  until  after  the 
turn  has  been  completed. 

3.  Feedback  may  be  absent.  For  example,  an  air  traffic  controller  may 
incorrectly  determine  that  there  is  conflict  between  two  aircraft,  when  there 
is  actually  no  conflict.  If  his  directions  to  the  aircraft  to  avoid  the  perceived 
conflict  do  not  actually  place  them  in  conflict  later  on,  there  will  be  no 
feedback  that  the  controller’s  assessment  of  the  situation  was  incorrect. 

4.  The  cost  may  be  excessive.  Training  tasks  may  involve  expensive  training 
equipment,  ranges,  ammunition,  or  simulation  of  opposing  forces  to  provide 
a  context  in  which  an  entire  task  may  be  performed. 

5.  Practice  may  not  be  sufficient.  Because  of  the  delays  that  are  inherent  in 
task  performance,  or  because  of  the  high  training  cost,  it  may  not  be 
possible  to  provide  the  amount  of  practice  required  for  proficiency. 

6.  The  mental  or  physical  workload  involved  in  performing  the  task  may  be  so 
extreme  as  to  inhibit  learning. 

Some  of  the  problems  described  above  may  be  solved  by  changing  the 
training  conditions  without  affecting  the  tasks  that  are  trained.  Changes  in 
training  conditions  might  decrease  the  realism  of  the  training  environment  to 
make  training  more  affordable,  enhance  feedback,  or  increase  the  efficiency  of 
training.  Thus,  we  may  train  job  tasks  in  a  trainer  that  represents  the 
environment  with  low  fidelity.  Similarly,  we  may  speed  up  or  slow  down 
simulated  time,  enhance  cues  or  performance  feedback,  or  train  in  conditions  that 
are  substantially  simpler  than  actual  battlefield  conditions.  All  of  these  options 
change  the  conditions  of  training,  making  them  different  from  job  performance 
conditions  so  that  training  might  be  more  efficient. 
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Another  approach  to  the  problems  changes  the  content  of  training  by 
focusing  training  on  the  skills  that  must  be  learned.  Skill  training  has  the 
capability  to  compress  the  time  required  to  train  critical  activities,  to  avoid 
duplication  of  training  when  several  tasks  have  common  skills,  and  to  reduce  the 
effect  of  extreme  workload  by  training  task  components.  Using  a  skill-based 
training  strategy,  it  may  be  possible  to  isolate  critical  skills,  and  to  provide 
substantially  more  training  on  these  skills  than  would  be  possible  if  the  skills 
were  embedded  in  a  task.  The  critical  skills  acquired  may  improve  the 
performance  of  many  tasks,  so  that  the  need  for  subsequent  task  training  is 
reduced.  Furthermore,  training  on  very  difficult  tasks  may  be  easier  when  the 
trainee  already  possesses  some  of  the  critical  prerequisite  skills.  Finally,  skill¬ 
training  strategies  may  require  a  less  realistic  representation  of  the  job 
environment  than  task-training.  We  expect  that,  under  the  proper  circumstances, 
the  use  of  skill-training  methods  can  improve  training  cost  effectiveness. 

However,  skill  training  also  has  its  disadvantages,  which  we  expect  would 
outweigh  the  advantages  in  some  circumstances.  First,  learning  the  individual 
skills  is  usually  not  sufficient  for  proficient  task  performance.  It  is  still  necessary 
to  provide  mission  training  to  coordinate  the  skills  properly.  Second,  skill  training 
in  a  context  that  is  sufficiently  different  from  the  real  battlefield  may  not  transfer 
to  task  performance.  Thus,  we  need  to  compare  the  conditions  of  skill  training  to 
actual  job  performance  conditions  to  ensure  transfer  of  training  to  the  job 
environment. 

Design  of  Skill-Based  Training  Systems 

The  previous  discussion  indicates  that  a  critical  aspect  of  designing  a 
training  system  that  incorporates  skill  training  is  determining  which  skills  should 
be  trained.  Other  problems  involve  designing  skill-based  training  devices  and 
determining  the  optimal  use  of  these  devices.  Also,  the  combined  use  of  skill- 
based  and  task-based  training  devices  in  an  overall  training  system  must  be 
determined.  Methods  for  training-system  design  should  incorporate  these 
considerations  in  designing  the  skill-based  component  of  the  training  system. 

Although  current  methods  for  medium  selection,  training-device  design,  and 
training-device  evaluation  often  consider  the  specific  skills  that  must  be  trained  in 
order  to  meet  training  requirements,  existing  models  for  training-device  design  are 
focused  at  the  task  level.  For  example,  the  OSBATS  (Optimization  of  Simulation- 
Based  Training  Systems;  Sticha,  Blacksten,  Buede,  Singer,  Gilligan,  Mumaw,  & 
Morrison,  1988)  model  is  principally  concerned  with  determining  the  task-training 
conditions,  and  does  not  consider  changes  in  training  content.  The  training 
strategies  addressed  by  the  OSBATS  model  assign  task  training  to  different 
training  conditions  represented  by  different  training  devices.  The  model  views 
each  training  activity  as  relating  to  a  single  task  or  training  requirement. 

In  a  task-based  analysis,  all  training  resources  are  assigned  to  tasks. 
Training  that  is  of  general  relevance  to  the  job,  but  which  is  not  relevant  for  a 
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specific  task,  such  as  classroom  training  that  provides  introductory  information,  is 
not  considered  by  task-based  models.  In  the  OSBATS  model,  each  task  has  an 
established  performance  requirement  that  must  be  satisfied  by  the  training 
system.  The  tasks  require  certain  cues  and  response  options  in  order  to  reach 
that  performance  level.  The  required  cues  and  response  options  are  used  by 
OSBATS  to  predict  how  effective  training  devices  of  different  levels  of  fidelity  will 
be  in  training  the  tasks.  Training-device  design  options  are  initially  screened 
according  to  how  many  task  cue  and  response  requirements  can  be  met  using  the 
design.  Ultimately,  the  design  options  are  evaluated  by  OSBATS  according  to 
which  options  can  be  used  to  meet  all  training  requirements  at  the  minimum  cost. 

The  design  logic  for  skill-based  training  devices  differs  from  that  for  task- 
based  simulators,  but  shares  some  of  the  same  principles.  The  logic  depends  on 
the  analysis  of  the  material  to  be  trained  into  components.  Critical  components 
are  the  underlying  general  abilities,  specific  skills,  prior  learning  specific  to  the 
situation,  and  sequencing  of  the  instructional  content.  A  model  is  required  to 
identify  these  skill-training  requirements,  to  design  a  device  to  train  the  skills  at 
minimum  cost,  and  to  determine  the  optimal  allocation  of  training  among  skill- 
based  and  task-based  training  devices  through  cost  and  benefit  tradeoffs. 

Organization  of  the  Report 

The  terms  ''skill"  and  "task"  have  received  several  different  definitions  in 
the  research  literature.  The  next  section  of  the  report  clarifies  our  understanding 
of  these  terms  and  their  implications  for  training  design.  It  discusses  how  skills 
are  defined,  how  they  are  distinguished  from  tasks,  what  types  of  skills  there  are, 
and  how  they  are  identified.  This  section  also  briefly  reviews  some  of  the  research 
on  the  effectiveness  of  skill  training. 

The  third  section  introduces  a  cost-effectiveness  framework  for  evaluating 
skill  training,  and  applies  this  framework  to  illustrate  the  rationale  behind  skill¬ 
training  strategies  as  well  as  the  conditions  under  which  these  strategies  should 
be  effective.  We  distinguish  three  ways  that  skill  training  can  be  used  to  improve 
training  cost-effectiveness.  First,  skill-based  training  can  be  used  to  isolate 
critical  skills  which  are  not  sufficiently  exercised  in  the  task  context.  The  proper 
skill  training  strategy  can  allow  the  student  far  more  practice  applying  these 
skills  than  would  be  possible  in  the  context  of  the  task.  Second,  skill-based 
training  can  provide  the  student  with  the  general  competencies  that  can  be 
applied  to  many  job  tasks.  Third,  skill  training  can  provide  the  component 
competencies  that  are  required  to  perform  very  complex  tasks.  This  kind  of  initial 
skill  training  can  reduce  the  workload  required  to  learn  the  tasks,  and 
consequently  increase  the  learning  rate. 

The  fourth  section  describes  our  model  for  the  design  of  skill-based  training 
systems  in  the  context  of  an  example  from  air  traffic  control  training.  This 
example  illustrates  the  four  major  design  processes  covered  in  the  model.  These 
processes  identify  skills  that  require  training,  develop  skill-training  strategies, 
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design  training  devices  to  implement  the  skill-training  strategies,  and  allocate 
training  resources  to  the  skill  training. 

The  fifth  section  contains  a  formal  system  description  of  the  model  for  skill- 
based  training  design.  The  description  includes  a  hierarchical  representation  of 
the  major  model  activities,  and  a  description  of  the  information  required  for  its 
use. 


The  final  section  provides  a  discussion  of  modeling  issues  that  could  affect 
the  formal  model  developed.  User  issues  are  raised  that  affect  implementing  a 
decision  aid  of  this  sort.  The  section  presents  a  discussion  of  research  issues,  and 
the  need  for  both  research  and  further  development  in  this  area. 
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SKILL  DEFINITION  AND  CHARACTERIZATION 


In  this  section,  we  provide  a  foundation  for  our  model  of  skill-based  training 
optimization  by  describing  the  notion  of  "skiH"  as  a  training  construct,  and  by 
outlining  a  method  for  identifying  skills  within  a  job  domain.  We  begin  with  a 
brief  review  of  the  history  of  skill  training.  In  it,  we  discuss  briefly  some  of  the 
more  recent  empirical  findings  related  to  skill  training.  We  then  discuss  the 
strengths  and  weaknesses  of  the  definitions  of  "task"  and  "skill"  as  currently 
applied  to  military  training,  and  propose  revisions  to  them.  Finally,  we  discuss 
the  kinds  of  skills  that  are  considered  in  the  model,  and  briefly  outline  an 
approach  to  identify  skills  for  training. 

Research  on  Skill  Training 

The  history  of  research  related  to  skill  acquisition  is  long  and  rich.  For 
example,  Adams’  (1987)  review  of  motor  skill  research  goes  back  nearly  a  century 
for  its  earliest  references.  Despite  the  diversity  of  the  literature  and  the  nearly 
universal  lack  of  consensus  on  a  single  definition  of  "skill,"  there  is  much  that  is 
consistent.  In  this  brief  review,  we  will  consider  several  descriptions  of  skills  and 
skilled  behavior.  We  will  also  review  several  methods  for  training  skills.  Our  goal 
is  to  provide  a  rationale  for  the  definition  of  skill  that  we  have  developed  to  guide 
the  development  of  our  skill-based  training  optimization  model. 

What  is  a  Skill? 


Adams  (1987)  suggests  that  the  roots  of  our  understanding  of  skill  go  back 
to  work  by  the  British  psychologist,  Pear,  in  1927.  "Pear  said  that  skill  has  an 
explicit  reference  to  the  quantity  and  quality  of  output.  A  skill  is  learned,  and  it 
is  distinguished  from  capacity  and  ability  because  an  individual  may  have  the 
capacity  and  ability  to  perform  a  skill  but  cannot  do  it  because  it  has  not  been 
learned"  (p.  42).  Summarizing  Pear’s  early  contributions  and  the  continued 
refinement  since  his  time,  Adams  identified  three  characteristics  that  define  skills 
(p.  42): 

1.  Skill  is  a  wide  behavioral  domain.  From  the  beginning,  skill 
has  meant  a  wide  variety  of  behaviors  to  analysts,  and  the 
behaviors  have  almost  always  been  complex. 

2.  Skill  is  learned.  Welford  (1968,  pp.  12-13)  said  that  "skill  is 
acquired  after  long  training,  and  consists  of  competent,  expert, 
rapid  and  accurate  performance." 

3.  Any  behavior  that  has  been  called  skilled  involves 
combinations  of  cognitive,  perceptual,  and  motor  processes  with 
different  weights.  Mathematicians  have  cognition  heavily 
weighted  in  the  description  of  their  behavior,  with  virtually  no 


weight  for  perception  or  the  motor  response  with  which  they 
write  the  answer  to  a  problem.  On  the  other  hand,  the 
behavior  of  tennis  players  could  not  be  meaningfully  described 
without  including  the  motor  responses  stemming  from  their 
perceptual  evaluation  of  the  situation  and  the  cognition  in  their 
decision  making. 

Distinguishing  skills  and  related  concents.  Fleishman  (1966)  draws  a 
distinction  between  abilities  and  skills.  He  uses  "ability"  to  refer  to  comparatively 
general  and  long  lasting  traits.  Although  "abilities"  are  enduring  and  difficult  to 
change,  they  are  learned.  They  develop  at  different  rates  and  most  of  the  learning 
occurs  during  childhood  and  adolescence. 

Skill  refers  to  the  level  of  proficiency  an  individual  exhibits  relative  to  a 
particular  task  or  group  of  related  tasks. 

Thus,  when  we  speak  of  acquiring  the  skill  of  operating  a  turret 
lathe,  we  mean  that  this  person  has  acquired  the  sequence  of 
responses  required  by  this  specific  task.  The  assumption  is  that  the 
skills  involved  in  complex  activities  can  be  described  in  terms  of  the 
more  basic  abilities.  For  example,  the  level  of  performance  a  person 
can  attain  on  a  turret  lathe  may  depend  on  his  basic  abilities  of 
manual  dexterity  and  motor  coordination.  However,  these  same  basic 
abilities  may  be  important  to  proficiency  in  other  skills  as  well  (p. 

148). 

Fleishman’s  more  recent  work  (eg.  Fleishman,  1982;  Mallamad,  Levine,  & 
Fleishman,  1980)  has  been  on  the  development  of  taxonomies  of  abilities  and 
skills.  He  has  developed  a  list  of  40  abilities  which  can  be  used  to  describe  skilled 
performance  on  tasks.  His  taxonomic  work  has  resulted  in  very  reliable  methods 
for  identifying  the  abilities  underlying  the  potential  level  of  skill  which  can  be 
attained  for  the  tasks  of  interest. 

While  Fleishman  has  focused  on  the  relationship  between  abilities  and 
skilled  performance  on  tasks,  Gagn6  (1975)  is  more  interested  in  the  outcomes  of 
learning. 

Learning  establishes  persisting  states  in  the  learner.  These  states 
make  possible  the  performances  that  are  observed.  Although  the 
performances  themselves  vary  in  many  dimensions,  the  underlying 
states  may  be  classified  as  having  certain  formal  properties  in 
common.  We  choose  here  to  call  these  persisting  states  "capabilities," 
a  word  which  implies  that  they  make  the  individual  "capable"  of 
certain  performances  ...  In  addition,  there  is  the  word  "capacity," 
which  traditionally  has  a  different  meaning,  namely,  the  innate  limit 
of  what  an  individual  can  learn.  According  to  this  usage,  an 
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individual  may  have  the  "capacity"  for  learning  certain  "capabilities;" 
but  what  he  learns  are  the  "capabilities"  (p.  50). 

Gagn6  does  not  provide  a  list  of  human  "capacities."  However,  he  does 
classify  "capabilities."  At  the  highest  level,  there  are  five  categories  of  learned 
capabilities:  Verbal  Information,  Intellectual  Skill,  Cognitive  Strategy,  Attitude, 
and  Motor  Skill.  The  Intellectual  Skill  category  is  subdivided.  The  most 
important  subcategories  for  practical  purposes  are:  Discrimination,  Concrete 
Concept,  Defined  Concept,  Rule,  Higher-Order  Rule.  Any  learning  task  may  be 
classified  according  to  Gagne’s  categories.  Ideal  conditions  for  learning  and 
recommendations  for  teaching  strategies  vary  from  category  to  category.  Hence, 
Gagn6  provides  a  framework  for  designing  instruction  that  should  facilitate 
learning  within  each  category  in  an  optimal  manner. 

Recent  research  in  cognitive  psychology  takes  a  somewhat  different  view  of 
skill,  skill  acquisition,  and  skilled  behavior.  Card,  Moran,  and  Newell  (1983)  view 
"skilled  behavior"  as  one  anchor  point  on  a  continuum  which  is  anchored  by 
problem  solving  at  the  other  extreme.  In  other  words,  one  cannot  talk  of  "a"  skill. 
Rather,  as  one  learns  to  negotiate  the  problem  space  underlying  a  task,  one  begins 
to  exhibit  skilled  behavior.  They  state: 

Unfortunately,  there  is  no  basis  for  constructing  a  general  taxonomy 
of  cognitive  skills.  Cognitive  skills  exist  for  all  cognitive  tasks  (ie.  all 
situations  that  permit  problem  solving),  provided  that  practice  on 
them  is  possible.  The  taxonomy  of  all  cognitive  skills  is  an  image  of 
the  taxonomy  of  all  possible  tasks  (p.  396). 

Stages  of  skill  acquisition.  While  it  may  be  true  that  each  cognitive  task 
requires  a  unique  cognitive  skill  for  expertise  to  be  demonstrated,  Anderson  (1985) 
argues  that  individuals  also  possess  general  problem  solving  procedures  in  order 
to  develop  skills  in  novel  domains.  Anderson  proposes  that  skill  development 
occurs  in  three  stages.  The  first  stage  is  called  the  cognitive  stage,  the  second 
stage  is  the  associative  stage,  and  the  final  stage  is  the  autonomous  stage. 

During  Anderson’s  first  stage,  learners  commit  to  memory  a  set  of  facts 
relevant  to  the  skill.  These  facts  are  called  declarative  knowledge.  Learners  use 
domain-general  problem  solving  procedures  and  declarative  knowledge  to  guide 
their  problem  solving.  The  most  common  domain-general  problem-solving 
procedures  are  heuristics.  Anderson  includes  difference-reduction  methods, 
means-end  analysis,  working  backward,  and  analogical  reasoning  as  examples  of 
such  domain-general  heuristics.  Analogical  reasoning  "attempts  to  use  the 
structure  of  the  solution  to  one  problem  to  guide  solutions  to  another  problem"  (p. 
218).  The  other  three  methods  are  all  variations  of  a  strategy  that  attempts  to 
define  subgoals  that  must  be  achieved  in  order  to  achieve  the  goal  of  solving  a 
given  problem. 
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During  the  associative  stage  two  important  phenomena  occur.  First,  errors 
are  detected  and  eliminated.  Second,  connections  among  the  various  elements 
necessary  for  performance  are  strengthened  and  smoothed.  This  process  involves 
the  transformation  of  declarative  knowledge  into  procedural  knowledge.  By  the 
end  of  the  associative  stage,  domain-specific  procedures  are  available  so  that  the 
learner  is  not  dependent  on  the  much  slower  combination  of  declarative  knowledge 
and  domain-general  problem-solving  procedures. 

In  the  autonomous  stage  of  skill  acquisition,  the  skill  becomes  more 
automated  and  rapid.  According  to  Anderson: 

No  sharp  distinction  exists  between  the  autonomous  and  associative 
stage.  The  autonomous  might  be  considered  an  extension  of  the 
associative  stage.  Because  facility  in  the  skill  increases,  verbal 
mediation  in  the  performance  of  the  task  often  disappears  at  this 
point.  In  fact,  the  ability  to  verbalize  knowledge  of  the  skill  can  be 
lost  altogether.  This  autonomous  stage  appears  to  extend 
indefinitely.  Throughout  it,  the  skill  gradually  improves  (p.  235). 

Anderson’s  work  suggests  that  a  rich  store  of  domain-general  problem¬ 
solving  procedures  would  facilitate  problem  solving  and  skill  acquisition  in  novel 
fields.  New  declarative  knowledge,  combined  with  the  appropriate  choice  of 
domain-general  procedures,  should  lead  to  the  development  of  domain-specific 
procedures.  Automatic  domain-specific  procedures  should  produce  skilled 
behavior. 

Recent  work  by  Perkins  and  Salomon  (1989)  is  supportive  of  the  view  that 
both  domain-general  procedures  and  knowledge  of  the  domain  are  necessary  for 
skill  acquisition.  Perkins  and  Salomon  view  general  cognitive  skills  as  "devices  for 
retrieving  and  wielding  domain-specific  knowledge"  (p.  23).  These  general 
cognitive  skills  exist,  and  they  always  function  in  contextualized  ways. 

Perkins  and  Salomon  do  not  have  a  list  of  cognitive  skills.  However,  they 
do  describe  the  characteristics  of  cognitive  skills  and  provide  examples.  A 
cognitive  skill  is  expected  to  exhibit  the  following  characteristics:  (a)  the  skill 
shows  seeming  use,  that  is,  experts  use  the  skill;  (b)  the  skill  appears  to  play  an 
important  role  in  task  reasoning;  (c)  the  skill  is  demonstrably  transferrable  and  is 
used  in  a  wide  variety  of  tasks;  and  (d)  the  skill  is  commonly  absent  in  novices. 
Examples  of  skills  include  looking  for  counterexamples,  using  analogies  and 
searching  for  misanalogies,  referring  to  intuitive  mental  models,  testing  extreme 
cases  of  potential  solutions,  and  constructing  simpler  problems  that  are  similar  to 
the  problem  of  interest. 

Studies  of  Skill-based  Training 

A  growing  body  of  research  has  shown  skill  training  in  various  forms  and 
across  several  applications,  to  be  a  viable  instructional  approach.  It  can  focus 
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attention  on  procedures  and  associated  skills  that,  while  important  to  overall 
performance,  are  not  captured  in  task-based  training.  It  can  foster  the 
development  and  automation  of  skills  more  efficiently  than  in  task-based  training. 
It  can  also  reduce  costs  by  demanding  lower  levels  of  physical  fidelity  in  the 
instructional  situation,  and  by  combining  training  on  several  skills  in  a  single 
program.  In  skill -training  environments,  for  example,  instructional  situations  or 
scenarios  need  not  correspond  to  job  tasks  to  provide  valuable  learning 
opportunities.  Scenarios  can  be  tailored  to  render  maximal  instructional  value, 
while  avoiding  the  high  cost  of  adhering  to  many  job-specific  constraints  (e.g., 
physical  and  functional  fidelity  of  the  job  environment).  Finally,  skill-based 
training  can  facilitate  transfer  of  knowledge  by  exposing  the  learner  to  the  utility 
of  a  skill  in  various  contexts. 

Part-task  training  research.  Much  of  the  research  on  skill  training  has 
been  conducted  under  the  heading  of  part-task  training  (PTT).  (For  a  detailed 
review  of  the  PTT  literature,  see  Knerr,  Morrison,  Mumaw,  Stein,  Sticha, 

Hoffman,  Buede,  &  Holding,  1986.)  The  PTT  approach  seeks  to  divide  instruction 
on  complex  jobs  into  manageable  parts,  and  can  be  more  effective  and  less  costly 
than  instruction  in  a  whole-task  format.  One  obstacle  to  the  application  of  PTT, 
however,  is  how  to  divide  up  the  job  to  achieve  these  benefits.  Naylor  (1962) 
proposes  three  methods  for  dividing  job  tasks  for  training:  simplification, 
segmentation,  and  fractionation. 

The  simplification  technique  adjusts  one  or  more  characteristics  of  a  task  to 
reduce  task  difficulty.  As  learning  progresses,  difficulty  is  systematically 
increased  to  more  closely  approximate  actual  task  parameters.  Segmentation 
partitions  tasks  into  independent  sub-tasks  along  temporal  or  spatial  dimensions. 
Sub-tasks  are  learned  separately  and  then  recombined  to  form  the  whole  task. 

Both  of  these  methods  suggest  a  task-based  approach  to  training. 

The  third  method,  fractionation,  is  more  appropriate  for  skill-based  training. 
The  fractionation  technique  divides  tasks  into  task  components  that  are  performed 
simultaneously  on  the  job.  Each  fraction  represents  a  psycho-motor,  perceptual,  or 
cognitive  competency,  or  skill,  that  the  learner  must  possess  to  perform  the  whole 
task  proficiently.  Once  identified,  fractions  are  trained  separately  before  being 
combined  in  whole-task  training. 

The  idea  of  practicing  skills  in  isolation  from  others  is  consistent  with 
several  learning  models.  Gagn6  (1968)  introduced  the  notion  of  learning 
hierarchies  of  intellectual  skills.  In  determining  how  to  sequence  instruction, 
capabilities  that  the  learner  must  have  to  perform  the  task  are  identified  and 
sequenced  according  the  order  in  which  they  become  relevant  in  performing  the 
task.  The  idea  is  that  certain  capabilities  are  prerequisites  for  others  and 
therefore  must  be  acquired  before  others  can  be  learned.  Fitts’  (1964)  work  on 
complex  skill  acquisition  divides  learning  into  three  stages:  cognitive,  associative, 
and  autonomous.  The  basis  of  a  skill  is  formed  in  the  first  stage  through 
conscious  mediation.  The  second  stage  establishes  and  refines  the  skill  through 
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practice  until  conscious  attention  is  no  longer  required.  The  third  stage  is 
characterized  by  increasing  speed  and  accuracy  in  applying  the  skill.  While  Fitts’ 
model  does  not  address  the  issue  of  training  directly,  the  idea  of  practicing 
individual  task  components  to  autonomy  has  implications  for  the  effectiveness  of 
skill-based  training.  The  role  of  fractionation  as  an  instructional  technique  can  be 
more  clearly  seen  in  recent  extensions  to  Fitts’  model.  Anderson,  Conrad,  and 
Corbett  (1989)  found  that  LISP  programming  skills  can  be  divided  into  units 
(production  rules)  that  are  learned  independently  of  one  another.  Presenting  the 
learner  with  problems  selected  to  practice  weak  skills  was  shown  to  improve 
learning.  Schneider  (1985),  in  his  work  on  training  perceptual  skills  emphasizes 
the  importance  of  teaching  component  skills  in  isolation  and  then  gradually 
combining  them  to  approximate  whole-task  performance.  (A  more  detailed 
description  of  Schneider’s  work  is  presented  below.) 

Based  on  this  work,  the  military  has  implemented  procedural,  cognitive, 
and  perceptual  pre-training  courses  to  help  students  develop  basic  skills  (Knerr,  et 
al.,  1986).  Procedural  skill  training  is  very  inexpensive  and  is  effective  for 
training  and  sustaining  skills  that  receive  little  practice  on  the  job  and  are, 
therefore,  susceptible  to  forgetting  (Adams,  1960).  Cognitive  pre-training  focuses 
on  knowledge  of  the  domain  that  is  thought  to  be  required  for  actual  task 
performance.  While  it  has  not  shown  effects  as  dramatic  as  the  other  two  forms  of 
pre-training,  it  has  been  shown  to  facilitate  initial  performance  in  job  task 
training  (Smith,  Waters,  &  Edwards,  1975).  Perceptual  pre-training,  the  most 
widely  studied  of  the  three,  focuses  on  prerequisite  skills,  such  as  recognizing 
patterns  or  cues,  that  must  become  automated  to  allow  for  time-sharing  with  more 
demanding  skills.  Training  in  perceptual  component  skills  has  been  demonstrated 
to  be  very  effective  in  aviation-related  jobs  (Schneider,  Vidulich,  &  Yeh,  1982; 
Schneider,  1985;  Myers  &  Fisk,  1987;  Fisk  &  Eggemier,  1988). 

Schneider  and  his  colleagues  have  identified  and  trained  skills  important  to 
the  job  of  air  intercept  control  (AIC).  AIC  can  be  broken  down  into  two  main 
classes  of  tasks:  (a)  collecting  data  about  the  current  situation,  and  (b) 
transmitting  those  data  to  aircraft  and  ship  weapons  controllers  (Halley,  Hooks, 
Lanford,  &  Nowell,  1981).  Two  of  the  skills  involved  in  these  tasks  are  monitoring 
a  radar  screen  and  calculating  trajectories.  These  superordinate  skills  are  not  at 
the  appropriate  level  of  specificity  for  initial  training,  and  must  therefore  be 
further  divided  into  component  skills. 

Schneider  (1985)  identified  the  component  skills  used  in  performing  a 
particular  sub-task  of  AIC  called  "co-speed  nearest  collision  intercepts."  This  task 
involves  eight  motor,  perceptual,  and  cognitive  skills.  For  example,  "heading 
identification"  involves  the  use  of  spatial  reasoning  skills  to  quickly  identify 
heading  angle  to  within  10  degrees.  "Basic  tracking"  skill  requires  fine  motor 
movements  to  accurately  position  a  cursor  over  a  radar  blip  twice  in  succession. 

All  eight  of  the  skills  can  be  initially  trained  independently,  and  then  combined 
into  an  orchestrated  superordinate  skill,  subtask,  or  whole  task  performance. 
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Based  on  this  analysis  Schneider  has  developed  a  computer-based  skill  trainer 
that  is  currently  used  to  train  the  AIC  job. 

The  methods  used  by  Schneider  (1985)  are  effective,  but  are  also  very 
resource  intensive.  Recently,  Fisk  and  his  colleagues  (Fisk  &  Eggemeier,  1988; 
Eggemeier,  Fisk,  Robbins,  &  Lawless,  1988)  developed  a  systematic  front-end 
analysis  methodology  for  identifying  component  skills  for  PTT.  The  method  uses 
interview  and  observation  techniques  to  identify  automatic  processes  that  are 
components  of  complex  skills.  It  has  been  used  to  identify  cognitive,  perceptual, 
and  motor  skills  that  underlie  operator  performance  in  aircraft  command-and- 
control  tasks. 

Application  of  computer  games.  Another  skill  training  paradigm  that 
appears  promising  is  the  application  of  computer  games  to  teaching  complex  skills. 
Gopher,  Weil,  Bareket,  &  Caspi  (1988)  designed  a  computer  game  to  simulate 
aviation-related  tasks  involving  psychomotor  and  cognitive  skill  components. 

Their  goal  was  to  facilitate  development  of  selective  attention  and  resource 
allocation  skills  in  complex  real-time  jobs,  such  as  piloting  an  aircraft,  in  a  low- 
cost  simulation  environment.  It  is  important  to  note  that  the  game  was  designed 
to  grossly  simplify  the  physical  environment  of  the  cockpit  (i.e.,  low  physical 
fidelity),  while  closely  approximating  the  skill  components  of  the  job  (i.e.,  high 
psychological  fidelity).  Their  approach  may  be  considered  PTT  in  that  the  game  is 
intended  to  train  certain  piloting  skills.  However,  their  method  deviates  from 
traditional  PTT  in  that  the  task  (the  game)  is  not  decomposed  into  its  parts  for 
training.  Rather,  the  task  is  presented  in  its  entirety,  and  the  training 
alternatively  emphasizes  the  need  to  direct  attention  and  allocate  resources 
according  to  changes  in  task  demands. 

The  effectiveness  of  the  training  was  tested  experimentally  by  inserting  the 
game  training  into  a  flight  training  curriculum.  Three  groups  of  student  pilots 
received  either  10  hours  of  game  practice  with  skill  instruction,  10  hours  of  game 
practice  without  instruction,  or  no  practice  between  two  phases  of  flight  school. 
Skill  transfer  was  measured  by  actual  performance  scores  in  the  latter  phase  of 
flight  training.  Results  indicate  that  subjects  who  received  game  practice  with 
instruction  scored  better  in  actual  flight  training  than  either  of  the  other  two 
groups.  Gopher,  et  al.  conclude  that  a  high  psychological  fidelity,  low  physical 
fidelity  trainer  can  be  effective  for  training  complex  skills  within  the  context  of  a 
whole-task  training  curriculum. 

Summary 

The  examples  cited  in  this  brief  review  cover  a  wide  range  in  both  time  and 
point  of  view.  They  use  different  vocabulary  and  suggest  different  levels  of  detail 
in  addressing  domain-specific  skilled  behavior  and  domain-general  problem  solving 
procedures.  However,  they  agree  in  the  sense  that  there  are  general  abilities, 
capacities,  or  domain-general  procedures  which  are  applied  to  the  development  of 
domain-specific  skills  and  skilled  behavior.  It  appears  to  be  valuable  to  consider 
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training  at  both  levels;  one  can  learn  domain-general  procedures  and  one  must 
learn  and  practice  domain-specific  skills.  At  also  appears  that  skill  training  is  a 
viable  and  effective  technique  for  developing  proficiency  in  complex  job  domains. 

In  designing  optimal  training  from  a  skills  point  of  view,  one  must  consider  the ' 
variety  of  levels,  the  generality,  and  the  specificity  that  can  be  implied  by  the  term 
’'skill.*'  In  the  next  section  we  propose  a  working  definition  of  skill  and  an 
approach  to  identifying  skills  that  is  derived  from  this  body  of  research. 

Distinguishing  Tasks  and  Skills 

In  this  section,  we  define  the  terms  "task"  and  "skill"  as  a  foundation  for 
developing  an  understanding  of  the  role  these  constructs  play  in  the  specification 
and  development  of  training  systems.  The  ability  to  distinguish  the  two  is  critical 
to  the  development  of  a  skill-based  training  model  that  can  be  used  by  those  other 
than  research  psychologists.  Our  characterization  of  tasks  and  skills  extends  the 
traditional  usage  of  the  terms  in  light  of  recent  theoretical  advances  in  the  study 
of  human  information  processing. 

Tasks 


We  begin  with  the  definitions  of  "task"  and  "skill"  contained  in  MEL-HDBK- 
220B,  Glossary  of  Training  Device  Terms  (1986):  "A  task  is  the  lowest  level  of 
behavior  in  a  job  that  describes  the  performance  of  a  meaningful  job  function." 

This  definition  reduces  performance  to  a  lowest  common  denominator,  the 
task,  at  a  level  unique  to  the  particular  job  under  consideration.  For  our 
purposes,  the  definition  is  inadequate  for  two  related  reasons.  First,  it  seems  to 
equate  the  function  or  goal  to  be  accomplished  with  the  procedure  employed  to 
accomplish  it.  Clearly,  one  can  specify  a  function  to  be  performed  without 
enumerating  the  steps  required  to  perform  it.  Conversely,  following  a  sequence  of 
steps  need  give  no  indication  of  its  function,  when  it  should  b°  performed,  or  why. 
One  characteristic  of  expertise  in  complex  technical  domains  is  the  availability  of 
multiple  strategies  or  procedures  to  accomplish  a  task,  each  of  which  may  rely  on 
a  different  kind  of  skill.  Expert  electronics  troubleshooters,  for  example,  use 
different  kinds  of  troubleshooting  strategies  (e.g.,  split-half,  historical, 
reconfiguration),  which  rely  on  various  cognitive,  perceptual,  and  motor  skills  such 
as  inductive  reasoning,  pattern  recognition,  and  dexterity  (Means  &  Gott,  1988). 

Second,  it  is  unclear  whether  the  same  behavior  or  procedure  applied  in  two 
different  contexts  (e.g.,  job  domains)  will  be  identified  as  a  "task"  in  both  contexts. 
The  key  phrase  in  the  definition  is  "meaningful  job  function."  To  define  "task"  it 
appears  that  one  must  first  define  the  term  "meaningful  job  function."  At  what 
point  does  a  behavior  become  meaningful?  This  question  must  be  answered 
independently  for  each  job  domain  under  consideration.  It  is  possible  that  in  one 
domain  a  procedure  is  deemed  too  insignificant  in  the  context  of  the  overall  job  to 
be  considered  a  meaningful  job  function,  while  the  same  procedure  is  considered 
integral  to  proficiency  in  a  second  domain.  Presumably,  a  task-based  approach 
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would  identify  the  procedure  as  a  candidate  for  training  in  the  latter  domain  only. 
While  this  is  not  inherently  undesirable,  it  can  inhibit  the  learner  from 
transferring  knowledge  from  one  context  to  another.  Even  within  a  job,  the  same 
procedure  may  apply  in  two  contexts.  Due  to  differences  in  overall  complexity 
between  the  two  contexts,  however,  the  procedure  may  only  be  identified  for 
training  in  one  context,  thereby  reducing  the  possibility  of  transferring  that 
knowledge  to  the  second  context. 

We  view  job  performance  as  a  process  of  establishing  and  satisfying  goals. 
One  way  to  facilitate  the  identification  of  tasks  in  a  meaningful  way  is  to  define  a 
task  in  terms  of  its  function,  or  the  goal  to  be  accomplished,  rather  than  the 
behavior  required  to  execute  it.  In  most  cases  the  job  performance  problem  space 
contains  a  hierarchy  of  goals  and  subgoals  that  correspond  to  tasks  and  their 
component  activities.  This  approach  separates  what  has  to  be  done  and  when  to 
do  it  from  how  it  gets  done.  Thus,  we  propose  the  following  definition  of  task: 

A  task  is  a  job-specific  goal  that  must  be  satisfied  or  function  that 
must  be  accomplished.  A  task  statement  describes  what  must  be 
done,  but  not  how  it  will  be  accomplished.  Tasks  can  be  sequential  or 
hierarchical  in  nature  and  can  exist  at  various  levels  of  specificity 
within  a  job  domain.  A  task’s  position  in  the  hierarchy  specifies  the 
conditions  under  which  (when)  it  applies. 


Skills 


By  separating  what  and  when  from  how,  we  allow  the  skills  used  in 
performing  tasks  to  be  defined  independently  from  tasks.  The  MIL-HDBK-220B 
definition  of  skill  is  the  following:  "A  skill  is  the  ability  to  perform  a  job-related 
activity  that  contributes  to  the  effective  performance  of  a  task." 

This  definition  contains  two  important  assumptions.  The  first  assumption 
is  that  skill  is  characterized  as  an  ability  possessed  by  the  performer;  it  is  not  the 
actual  set  of  behaviors  but  rather  the  competence  to  bring  the  appropriate 
knowledge  to  bear  in  executing  the  behaviors.  It  is  a  general  competency  that  can 
be  adapted  to  a  specific  domain.  This  suggests  that  skill  exists  in  two  states,  or 
stages.  In  the  first  stage,  skill  exists  independently  from  any  job  domain.  It  is 
defined  in  terms  of  domain-independent  attributes  of  the  human  information¬ 
processing  system.  These  include  motor,  perceptual,  and  cognitive  attributes.  The 
second  stage  is  the  domain-specific  competency  that  results  from  the  combination 
of  domain-independent  abilities  and  domain  knowledge.  This  characterization  is 
similar  to  Fleishman’s  (1967)  distinction  between  ability  and  skill.  Given  this  two- 
stage  definition,  it  should  be  possible  to  look  at  performance  on  a  specific  task  and 
identify  the  human  competencies  required  to  perform  that  task  proficiently  in 
terms  of  domain-independent  abilities  and  domain  knowledge. 

The  second  implicit  assumption  in  the  MIL-HDBK-220B  definition  is  that 
skills  are  associated  with  activities,  which  are  components  of  (and  therefore 
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smaller  than)  tasks.  This  characterization  corresponds  well  with  the  notion  of 
primary,  or  enabling,  skill.  However,  it  artificially  restricts  skills  to  a  certain  size 
or  level  of  specificity  that  is  dependent  on  the  level  at  which  tasks  are  defined. 

This  could  lead  one  to  the  conclusion  that  skills  cannot  be  treated  independently 
from  the  tasks  through  which  they  are  instantiated. 

This  conclusion  would  be  erroneous  for  two  reasons.  First,  it  suggests  that 
proficient  performance  acquired  outside  the  context  of  the  job  itself  may  not  be 
useful  on  the  job  since  it  is  not  tied  to  a  job  task.  We  argue  that  it  is  often 
valuable  to  consider,  and  indeed  teach,  skills  in  isolation  from  job  tasks.  Second, 
we  argue  that  performance  requiring  proficiency  on  a  number  of  related  tasks  in 
concert  can  be  considered  an  instantiation  of  a  single  skill.  Indeed,  the  restriction 
of  skill  to  describe  sub-task  level  behavior  introduces  the  danger  of  omitting  many 
higher-level  (i.e.,  cognitive)  skills  from  consideration.  Our  approach  identifies 
skills  at  several  levels  of  performance.  Thus,  our  definition  of  skill  in  a  job  context 
is  the  following: 

Skill  is  the  adaptation  of  human  abilities  to  a  particular  job,  in 
conjunction  with  domain-specific  knowledge,  resulting  in  performance 
that  accomplishes  a  job-related  goal. 

Identifying  Skills 

To  realize  the  promise  of  skill-based  training,  training  developers  must  be 
able  to  identify  appropriate  skills.  In  this  section,  we  will  outline,  in  broad  terms, 
an  approach  to  identifying  job  skills.  The  specific  steps  applied  in  a  given 
situation  will  vary  with  each  application.  For  example,  there  are  several  points  at 
which  the  identification  of  skills  can  be  initiated.  One  could  begin  with  a 
thorough  understanding  of  domain-independent  abilities  and  little  knowledge  of 
the  job  domain  (as  a  psychologist  would)  or  vice  versa  (from  a  subject-matter 
expert  point  of  view).  On  a  second  dimension,  one  could  begin  by  looking  at  the 
job  as  a  whole  or  at  individual  tasks.  We  provide  an  example  of  one  application  in 
a  later  section.  That  example  assumes  that  a  set  of  tasks  has  been  specified 
(through  task  analysis),  and  that  a  subject-matter  expert  will  be  employed  to 
identify  and  select  a  set  of  skills  as  candidates  for  skill-based  training. 

Our  approach  to  identifying  skills  focuses  on  three  categories  of  human 
competence  commonly  found  in  task  and  skill  taxonomies:  psychomotor, 
perceptual,  and  cognitive  (see  review  of  literature  in  Fleishman  &  Quaintance, 
1984).  Other  categories,  such  as  physical  and  social  competence,  have  been 
excluded  from  consideration  in  the  present  context.  Within  each  category  our 
approach  posits  "general  purpose"  information-processing  abilities  (Fleishman, 
1967).  A  tentative  list  general  abilities  for  each  category  is  shown  in  Table  1.  The 
list  was  compiled  from  several  sources,  including  Miller’s  (1971)  systems  task 
vocabulary,  Fleishman  &  Quaintance’s  (1984)  ability  categories,  and  Gagne’s 
(1974)  categories  of  learning  outcomes.  Definitions  of  the  abilities  are  provided  in 
the  Appendix.  The  critical  attributes  of  abilities  for  our  purposes  are  that 
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Ta^le  1 

List  of  General  Abilities 


Psychomotor  Abilities 

Control  precision 

Reaction  speed 

Dexterity 

Aiming 

Multilimb  coordination 

Response  orientation 

Hate  control 

Perceptual  Abilities 

Detection 

Pattern  recognition 

Selective  attention 

Visualization 

Spatial  reasoning 

Discrimination 

Cognitive  Abilities 

Verbal  comprehension 

Abstraction 

Quantitative  reasoning 

Computation 

Meta-cognition 

Time  sharing 

Inductive  reasoning 

Deductive  reasoning 

Analogical  reasoning 

Planning 

Prioritizing 

Knowledge  compilation 

Information  recall 

Forward  chaining 

Backward  chaining 

Categorization 

they  can  be  sharpened  with  practice,  and  they  provide  dimensions  that  can  be 
used  to  discriminate  expert  from  novice  behavior. 

Job  Skills 

Job  skills,  identified  within  a  particular  domain  (via  task  analysis),  are 
viewed  as  instantiations  of  abilities  applied  in  conjunction  with  domain-specific 
knowledge.  The  knowledge  component  of  a  job  skill  is  often  necessary  for  the 
identification  of  the  underlying  ability.  For  example,  in  air  traffic  control,  speed 
estimation  can  be  accomplished  several  ways,  including  computation,  spatial 
reasoning,  and  pattern  recognition.  Specific  knowledge  of  the  job  environment  and 
the  objects  in  that  environment  will  dictate  which  of  these  abilities  will  be  used  in 
a  given  situation. 

We  do  not  offer  a  job  skill  taxonomy  because  to  do  so  would  add  an  artificial 
layer  to  our  analysis  between  abilities  and  domain  knowledge.  Such  a  list  would, 
by  necessity,  be  at  a  level  that  spanned  many  jobs,  but  would  be  difficult  to  use  for 


17 


skill  training  purposes  in  any  one  job.  We  believe  it  is  more  useful  to  work 
directly  from  a  task  and  skills  analysis  to  identify  job  skills  and  their  underlying 
abilities. 

Skilled  Performance 


Skilled  performance  is  the  observable  manifestation  of  the  application  and 
refinement  of  abilities  in  a  specific  task  environment.  Hence,  one  determinant  of 
skilled  performance,  or  expertise,  is  the  amount  and  kind  of  domain-specific 
knowledge  the  performer  can  call  upon  in  a  given  situation.  Abilities  cannot  be 
applied  effectively  in  a  complex  domain  without  the  knowledge  to  drive  them.  We 
distinguish  among  three  types  of  domain-specific  knowledge:  factual,  procedural, 
and  strategic.  Factual  knowledge  is  information  about  concepts,  rules,  and  objects 
in  the  job  environment,  including  their  attributes  and  states.  Examples  are  the 
rules  of  use  between  two  airspace  sectors  set  out  in  a  letter  of  agreement  or  the 
performance  characteristics  of  a  DC- 10  aircraft.  Procedural  knowledge  is  knowing 
how  to  do  something.  The  steps  involved  in  managing  a  holding  pattern  stack  is 
one  example.  Strategic  knowledge  consists  of  domain-specific  goals,  models,  and 
strategies  that  mediate  performance,  especially  in  non-routine  situations  in  which 
no  simple  procedure  applies.  An  example  of  strategic  knowledge  is  a  plan  for 
sequencing  aircraft  arrivals  into  an  airport. 

Early  in  skill  acquisition,  knowledge  is  applied  in  an  inefficient,  effortful 
fashion  through  a  process  that  Anderson  (1983)  has  named  "interpretive" 
processing.  Even  simple  procedures  are,  at  first,  performed  by  following  a  discrete 
set  of  declarative  instructions.  Over  time,  through  repeated  application, 
declarative  knowledge  is  built  up,  or  compiled,  into  unitary  procedures  until  their 
application  requires  little  or  no  conscious  attention.  We  can  think  of  these  units 
at  productions,  or  if-then  rules,  that  contain  information  from  each  of  the  three 
knowledge  categories.  That  is,  a  production  would  contain  a  goal  to  be 
accomplished,  the  conditions  under  which  it  applies,  and  an  action  to  be  executed. 
When  compiled,  knowledge  is  applied  in  an  efficient,  all-or-none  fashion  when 
called  for,  and  is  typically  not  subject  to  errors.  Knowledge  compilation  allows  a 
technician,  for  example,  to  perform  a  diagnostic  procedure  without  having  to  think 
about  (or  look  up)  all  the  steps  involved.  It  is  the  application  of  knowledge  in  this 
compiled  state  that  we  generally  characterize  as  skilled  performance. 

By  defining  skill  as  the  adaptation  of  human  competencies  to  a  particular 
job,  we  have  eliminated  the  restrictions  placed  on  the  level  at  which  skills  can  be 
identified  by  the  specification  of  tasks.  We  can  now  speak  of  component  skills, 
associated  with  the  activity  (or  sub-task)  level  of  performance,  and  superordinate 
skills  at  the  task  and  function  levels.  The  confusion  between  skills  and 
procedures,  and  between  skills  and  tasks  is  also  alleviated.  Since  a  particular 
skill  is  defined  in  terms  of:  (a)  the  set  of  competencies  used  by  the  performer  in 
conjunction  with  (b)  domain-specific  knowledge  to  execute  (c)  a  procedure  that 
accomplishes  (d)  a  goal,  skill  cannot  simply  be  equated  with  tasks  or  procedures. 
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A  detailed  example  of  the  process  of  identifying  candidate  skills  is  presented 
in  fourth  section  of  the  report.  What  follows  is  only  a  brief  outline.  First 
candidate  skills  must  be  identified,  which  involves  a  hierarchical  decomposition  of 
tasks  into  successive  layers  of  sub-tasks  and  associated  job  skills  until  a 
"trainable"  job  skill  is  identified.  By  trainable  we  mean  that  the  skill  can  be 
defined  in  terms  of  the  domain-independent  abilities  and  domain-specific 
knowledge  required  in  performance,  as  defined  above.  In  the  language  of 
Instructional  Systems  Development  (ISD),  one  must  be  able  to  specify  a  clear 
learning  objective  and  precise  performance  criteria.  In  cognitive  theoretic  terms, 
one  must  be  able  to  specify  the  extent  knowledge,  input  information,  information 
processing  mechanisms  or  processes,  and  output  behavior.  Skills  expressed  at  a 
level  from  which  training  objectives  cannot  be  derived  must  be  decomposed  into 
their  component  skills.  For  example,  air  traffic  controllers  will  tell  you  that  the 
ability  to  "see  the  traffic"  is  the  most  important  skill  a  controller  must  have. 
However,  they  are  unable  to  specify  how  they  do  it,  and  more  importantly,  how  to 
train  students  to  "see"  traffic.  By  breaking  that  superordinate  skill  down  further, 
however,  we  find  several  perceptual  and  cognitive  component  skills  that  can  be 
trained.  These  skills,  then,  become  candidates  for  skill-based  training. 


Second,  after  identifying  the  component  skills,  the  instructional  developer 
must  determine  whether  they  are  appropriate  for  skill  training.  The  appropriate 
candidate  skill  should  display  the  following  characteristics:  (a)  it  should  be 
critical  to  accomplishing  the  task(s),  (b)  it  should  require  a  significant  amount  of 
effort  to  acquire,  (c)  it  should  be  applied  consistently  across  situations  within  the 
domain,  and  (d)  it  should  be  trainable  outside  the  context  of  actual  job  tasks.  In 
addition,  two  entry  conditions  should  exist  for  appropriate  skills.  Acquisition  of 
the  skill  should  not  require  significant  amounts  of  prerequisite  knowledge  to  train, 
and  should  not  be  dependent  upon  training  two  or  more  functions  concurrently 
(time-sharing). 

Summary 

The  approach  we  have  outlined  and  the  principles  it  embodies  are  described 
in  greater  detail  in  the  example  problem  and  formal  model  description.  It  is 
important  to  emphasize  that  we  do  not  propose  that  skill-based  training  supplant 
task-based  training.  We  view  the  two  as  complementary  components  of  an  overall 
training  system.  Task-based  training  provides  the  learner  with  an  understanding 
of  the  job  environment,  the  context  in  which  performance  takes  place.  Job-specific 
concepts  (e.g.,  functional  and  structural  relationships)  and  strategies  (what  to  do, 
when,  and  why)  are  developed  from  exposure  to  job  tasks.  Skill-based  training 
allows  the  learner  to  progress  more  efficiently  through  task  training  by  isolating 
and  providing  practice  on  individual  component  or  superordinate  skills.  In  jobs 
that  involve  real-time  performance,  such  as  air  traffic  control,  skill-based  training 
can  sharpen  skills  (from  keyboard  entry  to  prioritization  and  planning).  In  off-line 
jobs  such  as  electronics  troubleshooting,  learning  to  use  an  oscilloscope  or  to  read 
diagnostic  software  code  might  also  be  candidates  for  skill-based  training.  These 
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skills  must  be  applied  effortlessly  before  the  learner  can  effectively  orchestrate 
conceptual  and  strategic  knowledge  into  a  complete,  proficient  performance. 


The  next  section  presents  another  way  of  looking  at  the  benefits  of  skill- 
based  training,  focusing  on  cost-effectiveness  rather  than  psychological  arguments. 
It  is  important  to  note  that,  while  coming  from  divergent  perspectives,  the 
conclusions  drawn  from  the  two  viewpoints  are  consistent  with  respect  to  the 
feasibility  of  the  skill-based  approach. 
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RATIONALE  FOR  SKILL-BASED  TRAINING 


In  this  section,  we  describe  three  ways  that  skill-based  training  could 
improve  training  effectiveness:  (a)  Skill-based  training  could  be  used  to  isolate 
critical  skills  to  provide  more  training  on  these  skills  in  a  given  amount  of  time 
than  could  be  provided  if  the  skills  were  trained  in  the  context  of  the  task. 

(b)  Skill-based  training  could  be  used  to  train  skills  that  are  components  of  many 
tasks  required  for  successful  mission  performance,  (c)  Skill-based  training  could 
be  used  to  reduce  aH^ntional  demands  at  early  stages  of  training  of  complex  tasks. 
The  effects  of  these  three  factors  on  training  cost-effectiveness  is  enhanced  if 
training  the  skill  in  isolation  can  be  accomplished  using  a  cheaper  training  device 
than  would  be  required  to  train  the  task  in  which  the  skill  is  used. 

The  remainder  of  this  section  illustrates  how  these  benefits  of  skill-based 
training  could  occur  using  a  simple  mathematical  framework  based  on  learning 
curves.  The  general  framework  also  provides  a  way  to  discuss  the  costs  and 
benefits  of  different  training  strategies.  The  framework  is  developed  in  the  first 
subsection,  and  then  applied  to  illustrate  the  costs  and  benefits  of  skill-based 
training  strategies. 

Framework  for  Analyzing  Skill-Based  Training  Cost-Effectiveness 

The  goal  of  the  learning  framework  is  to  represent  how  well  and  how 
quickly  training  requirements  can  be  attained  as  a  result  of  various  training 
activities.  The  training  requirements  are  typically  represented  as  task 
performance  standards  in  the  context  of  a  mission  scenario.  Training  activities 
can  include  training  on  actual  equipment,  in  a  classroom,  or  using  a  training 
device,  simulator,  or  other  training  medium.  In  addition,  the  training  activities 
may  train  the  tasks  directly  or  may  be  focused  on  the  underlying  skills  required 
for  successful  mission  performance.  Our  learning  framework  uses  a  learning 
curve  to  describe  the  improvements  in  task  performance  that  result  from  a  single 
type  of  training  activity.  Cost-effective  training  activities  may  be  chosen  by 
comparing  the  learning  curves  for  different  training  activities  for  a  given  training 
requirement. 

Skill-based  training  strategies  may  be  represented  by  three  components,  as 
illustrated  in  Figure  1.  These  three  components  involve  (a)  original  skill  learning, 
(b)  transfer  of  training  to  the  tested  task  on  operational  equipment,  and  (c)  task 
training  to  meet  the  training  requirements  on  operational  equipment.  The 
example  shown  in  Figure  1  illustrates  the  effects  of  approximately  10  hours  of 
skill-based  training.  The  skill-based  training  produces  fairly  high  skill 
performance,  but  because  the  skill  is  only  one  component  of  the  task,  performance 
transfers  imperfectly  to  the  task.  However,  the  skill  training  replaces 
approximately  2.5  hours  of  training  on  the  task.  Therefore,  if  the  hourly  skill 
training  cost  is  less  than  25%  of  the  cost  of  task  training,  it  would  be  more 
efficient  than  task  training. 
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The  central  elements  of  the  framework  are  the  general  functional  form  of 
the  learning  curve,  the  characterization  of  transfer  of  training,  and  effects  of 
attentional  capacities  on  the  learning  rate.  From  these  three  factors,  we  may 
determine  the  slope  of  the  learning  curve,  and  calculate  cost-effective  skill-based 
training  strategies.  A  critical  assumption  of  this  simple  framework  is  that  the 
shape  of  the  task  learning  curve  is  not  affected  by  either  the  amount  or  pattern  of 
prior  skill  training.  That  is,  the  effect  of  prior  skill  training  on  task  learning  must 
be  summarized  by  the  task  performance  level  produced  by  the  skill  training. 
Problems  would  arise  in  determining  the  cost  tradeoffs  if  the  learning  curve  for 
the  task  was  distorted  by  the  acquisition  of  the  skill. 

Functional  Form  of  the  Learning  Curve 

For  many  activities,  a  power  function  provides  a  good  account  of  the 
relationship  between  performance  time  and  amount  of  practice  (Newell  & 
Rosenbloom,  1981;  Boff  &  Lincoln,  1988).  The  power  function  also  describes  skill 
acquisition  when  performance  is  assessed  by  other  measures,  such  as  errors 
(Newell  &  Rosenbloom,  1981).  The  following  equation  shows  the  most  general 
form  of  the  power  law  of  practice. 
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P(t)  =  A  +  C(S  +  t)\ 


(1) 


where  A  is  the  asymptotic  performance  level  as  the  amount  of  practice  (t) 
increases  without  limit,  S  is  the  amount  of  training  equivalent  to  prior  experience, 
and  C  and  k  are  learning  rate  parameters.  P  is  a  monotonically  decreasing 
function  of  its  argument,  t.  At  low  levels  of  t,  performance  improvements  (as 
measured  by  decreases  in  performance  time  or  errors)  occur  at  a  relatively  high 
rate;  as  the  amount  of  practice  (t)  increases,  the  rate  of  improvement  decreases. 

When  we  consider  measures  of  performance  that  increase  with  practice,  we 
need  to  reverse  the  direction  of  the  power  function.  That  is,  the  performance  must 
increase  with  practice,  rather  than  decrease  with  practice.  If  we  map  the  range  of 
possible  performance  levels  to  the  interval  [0,1],  then  we  may  represent  the  power 
law  of  practice  with  the  following  equation: 

P(t)  ■  1  -  [1  +  CCS  +  t)]k.  (2) 

The  functions  shown  in  Figure  1  illustrate  the  shape  of  the  resulting  power 
function. 

Transfer  of  Training 

Since  possessing  a  specific  skill  is  just  one  component  of  competent  task 
performance,  we  would  expect  skill  training  to  transfer  imperfectly  to  task 
performance.  Figure  1  expresses  transfer  of  training  as  a  linear  function, 
consistent  with  Cronholm  (1985).  The  degree  of  transfer  of  training  is  represented 
by  the  slope  of  the  function.  Because  we  represent  transfer  of  training  as  a  linear 
function,  we  may  easily  combine  the  learning  and  transfer  functions  to  obtain  the 
following  composite  function  that  expresses  task  performance  as  a  function  of  skill 
training: 

P(t)  =  A{1  -  [1  +  C(S  +  t)]  k).  (3) 

The  asymptote  of  this  function,  A,  is  the  slope  of  the  transfer  of  training 
function.  We  expect  that  the  asymptote  will  depend  on  how  critical  the  skill  is  in 
determining  task  performance.  Training  on  very  critical  skills  will  produce  a 
composite  learning-transfer  function  with  an  asymptote  near  1.0;  training  on  less 
critical  skills  will  produce  a  function  with  a  lower  asymptote.  The  composite 
learning-transfer  functions  that  correspond  to  the  functions  shown  in  the  previous 
example  are  shown  in  Figure  2.  S  and  k  are  set  to  0.0  and  0.7,  respectively,  for 
both  curves.  For  the  task  training  curve,  C  =  0.5,  and  A  =  1.0.  For  the  skill 
training  curve,  C  =  0.6,  and  A  =  0.6. 

Learning  Rate 

The  learning  rate  of  the  power  function  depends  on  two  parameters  of  the 
function,  C  and  k.  Card,  Moran,  and  Newell  (1983)  have  suggested  that  the 
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exponent  is  relatively  constant;  we  will  assume  that  k  =  0.7  for  this  discussion. 
Thus,  in  the  application  of  these  functions,  changes  in  the  learning  rate  are 
represented  by  changes  in  C.  Changes  in  the  learning  rate  may  reflect  training 
strategies  that  make  training  more  or  less  efficient.  For  example,  a  skill-based 
training  strategy  may  allow  more  training  trials  to  occur  in  a  given  amount  of 
time,  leading  to  a  higher  value  of  C.  In  addition,  the  use  of  appropriate 
instructional  features  may  make  training  trials  more  effective,  again  increasing 
the  value  of  C. 


A  more  complicated  change  in  the  learning  function  may  occur  when  very 
difficult  tasks  are  being  trained.  This  change  will  occur  if  the  degree  of  difficulty 
exceeds  the  learning  capacity  of  the  student.  In  other  words,  when  a  task  is  very 
difficult  it  may  require  more  attentional  capacity  than  is  possessed  by  the  student, 
Thus  early  learning  may  be  slower  than  would  be  indicated  by  the  baseline  power 
function.  However,  as  performance  on  the  task  improves,  it  begins  to  require  less 
attention.  At  some  point  the  attentional  requirements  of  the  task  are  within  the 
capabilities  of  the  student,  and  learning  proceeds  at  its  nominal  rate. 


This  simple  representation  of  the  role  of  attentional  capacity  in  learning 
does  not  distinguish  the  cause  of  the  learning  difficulty.  At  least  three  causes  are 
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plausible:  (a)  workload  may  be  high  because  of  the  need  to  perform  several  task 
elements  simultaneously;  (b)  task-element  performance  may  interfere  with 
rehearsal  processes,  or  (c)  sequential  links  between  task  elements  may  prevent 
successful  performance  of  later  task  elements  without  the  cues  provided  by 
completion  of  earlier  task  elements.  Knowledge  of  the  cause  of  high  attentional 
demands  may  affect  the  choice  of  skills  for  training  or  the  design  of  training 
devices  for  skill  training.  However,  the  immediate  purpose  of  this  discussion  is  to 
provide  a  tentative  description  of  the  effects  of  attentional  limitations  on  learning, 
rather  than  to  determine  their  causes.  Furthermore,  determining  the  causes  of 
attentional  demands  will  require  considerable  additional  research.  Hence,  we  will 
not  distinguish  these  three  possible  causes  in  the  following  discussion. 

Our  formal  treatment  of  the  effects  of  attention  limits  is  similar  to  that  of 
Knerr,  Morrison,  Mumaw,  Sticha,  Blacksten,  Harris,  and  Lahey  (1987).  We  begin 
by  examining  the  derivative  of  Equation  2  (to  simplify  the  presentation,  we 
assume  that  the  asymptote  of  the  learning  function  is  1.0), 

P'(t)  =  kC[l  +  C(S  +  t)]**”.  (4) 

Algebraic  manipulation  of  Equation  2  allows  us  to  determine  that  [1  +  C(S  +  t)]  is 
equal  to  (1  -  P)'1*.  Hence,  we  may  rewrite  Equation  4  as  follows: 

PXt)  =  kC(l  -  P)^1*  (5) 

Equation  5  represents  the  slope  of  the  learning  function,  P'(t),  as  a  function  of  the 
performance  level,  P. 

In  the  following  development,  we  define  a  learning  function,  Q(t),  with  a 
derivative  that  is  the  same  as  P'(t)  when  the  attention  requirements  of  the  task 
are  within  the  capabilities  of  the  student,  but  that  has  a  derivative  that  is  less 
than  P'(t)  when  the  task  requires  capabilities  greater  than  those  possessed  by  the 
student.  We  will  denote  the  task  attention  requirement  by  R,  and  the  student 
attention  limit  by  L.  Then  we  will  define  Q(t)  as  a  function  such  that 

Q'(t)  =  kCftl  -  Q)<*+iy\  (6) 

where 

fa  f  L/[R(1  •  Q)]  if  Q  <  1  -  IVR 
\  1  otherwise. 

The  function,  f,  is  always  less  than  1.0.  Assuming  that  L  <  R,  f  has  the  value  L/R 
when  Q  =  0.0.  As  performance  (Q)  improves,  f  increases  correspondingly,  as  does 
the  learning  rate  (since  f  is  a  multiplicative  factor  of  Q').  When  Q  increases 
further  to  1  •  IVR,  then  f  =  1,  and  the  Q  has  the  same  slope  as  P.  As  Q  increases 
further,  the  value  of  f  remains  1. 
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We  will  only  be  concerned  with  the  first  case,  (Q  <  1  *  IVR).  By  combining 
the  definition  of  f  into  Equation  6,  we  obtain  the  following  equation. 

Q'(t)  =  (kCL/RXl  -  Q)*  (7) 

Equation  7  is  a  first  order  differential  equation  in  which  the  variables  can  be 
separated.  This  equation  can  be  solved  for  Q  to  yield: 

Q(t)  =  1  -  (1  +  [(1  -  k)LC/R3(t  +  S)}^”  (8) 

Equation  8  shows  that  Q(t)  is  a  power  function  of  the  same  form  as  P(t) 
(Equation  2),  but  with  different  learning  rate.  Figure  3  compares  the  two 
functions,  assuming  that  the  attention  requirement  is  three  times  greater  than  the 
student’s  capability  (i.e.,  R  =  3,  L  »  1).  Other  parameters  have  the  same  values  as 
the  task  learning  curve  shown  in  Figure  2  (i.e.,  C  =  0.5,  k  =  0.7,  A  =  1.0).  Under 
this  condition,  training  is  considerably  slower  than  what  would  be  the  case  if  there 
were  effects  of  attention  capacity.  It  takes  12  hours  to  accomplish  the  training 
that  would  have  required  about  7.5  hours  if  there  were  no  attention  effects. 
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It  should  be  noted  that  the  specific  form  of  the  function,  f,  in  this 
development  is  hypothetical.  We  know  of  no  research  that  has  attempted  to 
assess  the  form  of  this  function;  consequently,  we  have  used  the  simplest  form. 
Some  research,  such  as  that  of  Schneider  and  ShifTrin  (1977;  Shiffrin  and 
Schneider,  1977),  suggests  that  attentional  requirements  may  decrease  at  a  slower 
rate  than  is  suggested  by  f.  These  results  would  suggest  an  even  bigger  effect  of 
attention  limits  on  learning  rates.  Such  results  could  be  incorporated  by  making 
the  function,  f,  a  power  function.  However,  even  in  its  simpler  form,  this  example 
illustrates  some  of  the  possible  effects  of  changes  in  the  learning  process  due  to 
attentional  capacities. 

Incorporating  Cost  Into  the  Composite  Functions 

The  development  so  far  has  concentrated  on  performance  as  a  function  of 
training  time.  However,  we  may  easily  describe  performance  as  a  function  of 
training  cost,  if  there  is  a  constant  hourly  training  cost.  The  hourly  training  cost 
may  incorporate  an  allocation  of  fixed  cost  items,  as  well  as  variable  cost 
components.  If  h  represents  the  hourly  training  cost,  and  d  represents  the  total 
training  cost,  then  we  may  rewrite  Equation  3  to  represent  performance  as  a 
function  of  cost  as  follows: 

P(d)  =  A{1  -  [1  +  (C/hXS'  +  d)]k),  (9) 

where  S'  is  the  cost  equivalent  to  previous  experience.  Equation  8  may  be 
rewritten  similarly. 

The  assumption  of  constant  hourly  training  cost  is  a  simplification,  and  does 
not  allow  us  to  consider  such  factors  as  economies  of  Beale  or  discrete  costs 
involved  with  the  procurement  of  training  devices.  Nevertheless,  within  a 
restricted  range  of  training  hours,  the  assumption  is  probably  reasonably  accurate. 
Furthermore,  for  the  purposes  of  this  discussion,  we  prefer  the  simpler 
characterization  of  training  cost,  because  we  are  not  interested  in  training 
strategies  that  depend  on  details  of  the  training  cost  function  for  their 
justification. 

The  Slope  of  the  Cost-Effectiveness  Functions 

The  optimal  training  strategy  should  maximize  the  gain  in  performance 
obtained  from  the  training  cost.  That  is,  the  training  strategy  should  be  based 
upon  the  derivative  of  Equation  9  for  each  training  option  under  consideration.  As 
the  previous  development  has  illustrated  (in  Equation  5),  the  derivative  of  the 
learning  function  can  be  expressed  as  a  function  of  the  performance  level.  The 
derivative  of  Equation  9  is  shown  below: 

P'(d)  =  (AkC/h)(l  -  P/A )(k*lVk.  (10) 
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Figure  4  illustrates  this  function  for  hypothetical  skill-based  and  task-based 
training  options  (S  =  0.0,  k  =  0.7,  h  =  1.0).  The  skill-based  option  approaches  a 
lower  asymptote  (A  =  0.6)  than  the  task-based  option  (A  =  0.1),  but  it  approaches 
the  asymptote  at  a  higher  rate  (C  =  0.6  vs.  0.1),  perhaps  because  it  compresses  the 
training  time  required  or  because  it  requires  a  less  expensive  training  device.  The 
graph  makes  the  optimal  choice  obvious;  skill  training  should  be  chosen  at  low 
performance  levels,  while  task  training  should  be  chosen  at  high  performance 
levels.  The  remainder  of  this  section  specifies  some  of  the  conditions  in  which 
skill-training  options  might  be  optimal.  _ 


Marginal  Effectiveness/Cosi  Ratio 
0.3 

0.25 

0.2 


0.15 

0.1 

0.05 

0 

0  0.2  0.4  0.6  0.8  1 

Performance  Level 


Figure  4.  Marginal  cost-effectiveness  as  a  function  of  performance  level. 


Characterization  of  Skill-based  Training  Benefits 

The  framework  presented  above  provides  a  way  to  illustrate  the  benefits 
that  can  occur  from  skill  training.  In  addition,  our  analysis  of  skill -training 
benefits  will  enlighten  us  regarding  the  conditions  under  which  the  benefits  of 
skill  training  should  occur.  We  will  use  the  framework  to  address  possible  three 
benefits  of  skill  training.  First,  skill  training  can  provide  much  more  practice  on  a 
critical  skill  in  a  given  amount  of  time  or  for  a  given  cost.  Second,  critical  skills 
can  generalize  to  many  tasks,  so  that  training  them  can  avoid  unnecessary  and 
redundant  task  training.  Third,  training  the  critical  skills  involved  in  complex 
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and  difficult  tasks  can  decrease  the  mental  workload  required  to  perform  the 
tasks,  and  consequently  speed  up  subsequent  task  learning.  We  will  discuss  each 
of  these  benefits,  and  will  present  graphs  of  that  illustrate  the  types  of  learning 
and  cost-effectiveness  function.'  that  can  produce  the  benefit.  Finally,  we  will 
enumerate  some  of  the  conditions  that  should  maximize  the  likelihood  that  the 
benefits  will  be  obtained. 

Time  Compression  and  Reduced  Training  Cost 

Some  tasks  are  structured  so  that  there  is  little  opportunity  to  practice 
critical  skills.  For  example,  one  of  the  critical  skills  in  air  traffic  control  involves 
specifying  the  point  where  an  aircraft  should  turn.  As  Schneider  (1985)  has 
pointed  out,  the  skill  required  to  specify  the  turn  point  correctly  is  very  difficult  to 
learn,  both  because  of  perceptual  decay,  and  because  the  turn  takes  so  long  to 
perform  that  the  student  can  only  obtain  a  limited  number  of  repetitions  in  a 
given  amount  of  time.  However,  Vidulich,  Yeh,  and  Schneider  (1983)  have  shown 
that  by  isolating  the  skill  in  training,  it  is  possible  to  get  an  increase  of  between 
one  and  two  orders  of  magnitude  in  the  number  of  practice  iterations  that  are 
possible  within  a  specified  time. 

Figure  5  illustrates  the  effect  of  a  twenty-fold  O’ust  over  one  order  of 
magnitude)  increase  in  the  number  of  trials  per  unit  of  time  on  the  learning  and 
cost-effectiveness  functions  (with  S  =  0.0,  k  =  0.7,  h  =  1.0).  The  effect  of 
increa  ing  the  number  of  trials  per  unit  time  was  accomplished  by  increasing  the 
learning  rate  parameter,  C,  of  the  skill-learning  function  from  0.5  to  10.0.  The 
asymptote  of  the  skill-training  curve  was  set  at  0.3  to  indicate  that  although  the 
skill  is  critical  for  the  task,  it  represents  only  one  component  ability  required  to 
perform  the  task.  Nevertheless,  the  cost-effectiveness  curve  in  Figure  5b  indicates 
the  superiority  of  skill  training  below  a  performance  level  of  0.18,  which  is  60%  of 
the  asymptote  of  the  skill -training  function. 

The  benefit  of  compression  of  training  time  may  be  enhanced  if  the  skill 
training  can  be  accomplished  using  a  training  device  that  is  substantially  less 
expensive  to  procure  and  operate  than  a  training  device  required  for  task  training. 
Here  again,  orders-of-magnitude  improvements  are  possible.  In  the  previous 
example,  if  the  skill  training  could  be  conducted  on  a  training  device  that  cost  10% 
as  much  as  a  task  trainer,  then  skill-based  training  would  be  optimal  up  to  86%  of 
the  asymptote  of  the  skill -training  function,  or  a  task  performance  level  of 
approximately  0.26. 

Time  compression  can  be  a  very  powerful  benefit  of  skill-based  training. 

The  power  of  this  benefit  comes  from  the  fact  that  high  levels  of  time  compression 
may  be  possible.  Several  considerations  must  be  made  of  the  types  of  skill  that 
must  be  possessed  for  successful  job  performance.  Some  of  the  factors  that  are 
required  to  obtain  this  type  of  benefit,  or  to  enhance  it,  are  listed  below. 
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a.  Task  and  skill  teaming  functions. 

b.  Co st effectiveness  function* 
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Figure  5.  Learning  and  cost-effectiveness  functions  illustrating  training  time 

compression. 

1.  The  effect  of  skill  compression  is  enhanced  if  the  skill  being  trained  is 
critical  to  a  task.  High  criticality  leads  to  a  high  asymptote  of  the  learning 
function,  which,  in  turn,  increases  the  marginal  cost  effectiveness  as 
calculated  in  Equation  10. 

2.  The  effect  of  skill  compression  is  enhanced  if  the  initial  skill  level  is  low. 

As  Figure  5  illustrates,  the  advantage  of  skill  training  over  task  training 
decreases  as  a  function  of  performance  level.  Thus,  the  benefits  of  skill 
training  are  maximized  at  low  skill  levels.  At  higher  performance  levels 
cost-effectiveness  of  skill  training  suffers  because  of  imperfect  transfer, 
related  to  the  need  to  integrate  skills. 

3.  Effective  skill  training  requires  that  the  skill  can  be  isolated.  For  a  skill  to 
be  trained  in  isolation,  there  must  be  a  consistent  mapping  between  stimuli 
and  responses.  By  consistent  mapping,  we  mean  that  the  same  stimulus 
situation  produces  same  response.  Since  performance  of  activities  with  a 
insistent  mapping  does  not  depend  on  context  variables  the  skill  can  be 
trained  outside  of  the  job  context. 
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4.  The  benefit  of  time  compression  is  that  time  can  be  saved  by  skill  training. 
An  order  of  magnitude  improvement  in  training  time  is  desired;  otherwise, 
advantages  may  be  lost  due  to  imperfect  transfer.  A  good  indicator  of 
candidate  skills  for  time  savings  are  those  in  which  feedback  is  delayed.  If 
skill  training  can  reduce  the  delay  period  in  which  the  learner  is  awaiting 
feedback,  then  not  only  will  a  greater  number  of  training  trials  be  possible 
within  a  period  of  time,  but  the  trials  are  likely  to  be  more  effective  because 
feedback  is  more  immediate. 

6.  The  benefit  from  time  compression  is  enhanced  if  the  skill  can  be  trained 
with  a  training  device  that  is  less  costly  than  the  device  normally  required 
to  train  the  task.  Fidelity  requirements  for  skill  training  must  be  inferred 
by  an  analysis  of  the  activities  by  which  a  skill  is  instantiated,  much  as  was 
done  for  tasks  by  the  OSBATS  models  (Sticha,  Blacksten,  Buede,  Singer, 
Gilligan,  Mumaw,  and  Morrison,  1988).  We  expect  that  since  -ki11  training 
isolates  the  skill  from  the  job  context,  and  thus  allows  a  much  simplified 
representation  of  the  job  situation,  it  will  usually  require  a  less  expensive 
training  device. 

Generalization 


A  second  reason  for  skill  training  is  that  skills  may  generalize  to  several 
tasks,  functions,  and  even  new  jobs.  Thus,  training  several  well  chosen  skills  may 
enhance  the  performance  of  many  job  functions.  The  generalization  of  skills  to  job 
functions  is  somewhat  more  complex  than  time  compression.  In  order  to 
characterize  skill  generalization,  we  need  to  make  several  assumptions  regarding 
how  training  will  transfer  among  tasks  and  between  skills  and  tasks.  In  our 
example,  we  first  consider  a  single  task  and  a  single  skill.  Then  we  generalize  the 
analysis  to  cover  a  single  skill  and  several  tasks. 

Our  illustration  of  the  benefits  of  skill  generalization  is  based  on  the 
following  four  assumptions:  (a)  A  skill,  as  a  single  component  of  a  task,  transfers 
imperfectly  to  task  performance.  Thus,  the  asymptote  of  the  curve  relating  skill 
training  to  task  performance  is  less  than  one,  depending  on  the  relative  criticality 
of  the  skill  to  the  task;  (b)  since  a  skill  is  a  single  component  of  a  task,  it  should 
be  learned  faster.  That  is,  the  learning  rate  should  be  higher  for  skill  training 
than  for  task  training;  (c)  because  skill  training  does  not  incorporate  the  task 
performance  context,  the  reduction  in  the  asymptote  will  be  greater  than  the 
increase  in  the  learning  rate;  (d)  skills  can  be  learned  during  task  training  and 
transferred  from  one  task  to  another.  However,  transfer  of  a  skill  acquired  in  the 
context  of  a  task  should  be  less  than  the  transfer  of  training  from  skill-based 
training  to  task  training.  This  result  is  expected  because  the  skill  training  should 
produce  more  general  learning  that  can  be  applied  to  a  wider  variety  of  situations 
than  task  training  will  produce. 

Illustration  of  assumptions  for  a  single  task.  The  first  three  assumptions 
about  skill  generalization  can  be  illustrated  using  a  single  task.  First  assume  that 
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a  single  skill  accounts  for  one-third  (33%)  of  task  performance.  The  skill  training 
curve  is  obtained  from  the  task  training  curve  by  increasing  the  learning  rate,  C, 
from  0.5  to  1.5  (assuming  that  the  skill  is  learned  roughly  three  times  more  easily, 
see  b,  above).  We  also  reduce  the  asymptote,  A,  to  account  for  assumptions  (a) 
and  (c)  above.  The  decrease  in  A  will  be  greater  than  the  increase  in  C  because 
there  are  two  factors  affecting  the  asymptote,  while  there  is  a  single  factor 
affecting  the  learning  rate.  In  the  example,  we  set  A  =  0.3  for  the  skill-training 
curve,  which  is  90%  of  what  it  would  have  been  if  the  difference  in  the  training 
context  between  skill  training  and  task  training  had  no  effect  (0.33,  assuming  that 
the  skill  comprised  one-third  of  the  task  performance).  Because  we  are 
investigating  the  effects  of  generalization,  we  will  assume  that  there  is  no  time 
compression.  In  determining  the  cost-effectiveness  functions,  we  will  simplify  the 
problem  and  assume  that  the  hourly  training  cost  is  the  same  for  the  two  options. 

The  single-task  learning  and  cost-effectiveness  curves  shown  in  Figure  6 
indicate  that  there  is  no  advantage  to  skill  training  at  any  performance  level  for 
this  example.  When  the  performance  is  at  a  minimal  value  (0.01),  the  slope  of  the 
skill  learning  function  is  90%  that  of  the  task  learning  function,  reflecting  the 
effects  of  the  difference  in  training  context  between  skill  training  and  task 
training  (assumption  c).  As  performance  increases,  the  slope  of  the  skill  training 
curve  decreases  faster  than  that  of  the  task  training  curve  because  of  its  lower 
asymptote. 

Illustration  of  generalization  for  multiple  tasks.  Now  we  investigate  the 
case  in  which  the  same  skill  is  involved  in  several  tasks.  We  consider  two 
alternative  training  alternatives:  (a)  training  a  skill  in  a  general  context  so  that  it 
will  transfer  to  several  tasks,  or  (b)  training  a  skill  in  the  specific  contexts 
provided  by  the  job  tasks.  For  this  illustration,  we  assume  there  are  five  tasks 
that  require  a  given  skill,  and  that  the  skill  accounts  for  33%  of  the  performance 
of  each  of  five  tasks.  We  assume  that  in  all  other  respects,  the  tasks  are 
completely  unrelated.  Since  learning  the  unique  portion  of  each  task  is  not 
affected  by  skill  training,  we  will  not  consider  it  in  the  analysis.  Rather,  we  will 
focus  on  the  33%  that  is  common  to  all  five  tasks.  Thus,  this  part  of  the  example 
will  not  address  the  first  two  assumptions  made  above. 

In  keeping  with  assumption  (c)  made  in  the  single-task  case,  we  will 
assume  that  skill  training  transfers  90%  to  the  common  portion  of  the  task.  Since 
we  are  restricting  our  consideration  to  this  common  portion,  we  set  the  asymptote 
of  the  skill  learning  and  transfer  function  to  0.9.  The  asymptote  is  set  to  0.9, 
rather  than  0.3  because  we  are  ignoring  the  task-specific  portions  of  each  of  the 
five  tasks.  This  assignment  means  that  when  we  train  the  specific  skill  in  a 
general  context,  the  learner  is  90%  of  the  way  toward  applying  that  skill  in  five 
specific  contexts.  Other  parameters  of  the  skill  learning  and  transfer  function  are 
the  same  as  was  used  in  the  single-task  case  (C  =  1.5,  S  =  0.0,  k  =  0.7). 

Assumption  (d)  of  this  analysis  states  that  transfer  of  a  skill  from  one 
specific  context  (or  task)  to  another  specific  context  will  be  less  than  the  transfer 
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Figure  6.  Single-task  learning  and  cost-effectiveness  functions  assuming  no  time 
compression  or  reduced  training  cost. 

of  skill  from  a  general  context  to  a  specific  context.  One  way  to  arrive  at  an 
estimate  of  the  parameter  for  transfer  between  tasks  is  to  assume  that  transfer 
between  tasks  is  represented  by  a  two-step  process.  We  start  by  assuming  that 
the  student  learned  the  skill  in  a  task-specific  context.  The  first  step  re-encodes 
skilled  performance  in  a  general  way  based  on  the  additional  task  performance 
requirements.  Second,  the  general  skill  obtained  by  task  training  must  be 
transferred  to  the  new  task.  We  assume  that  90%  of  the  skill  learned  in  the 
original  task-specific  context  will  transfer  to  the  general  (multi-task)  context.  The 
second  step  is  the  same  as  transfer  from  skill  training  to  task  training,  so  we 
assume  that  90%  of  the  skill  transfers.  These  assumptions  do  give  an  inherent 
advantage  to  skill  training,  but  not  an  overwhelming  advantage.  We  can  then 
estimate  the  overall  transfer  from  one  specific  context  to  another  as  the  product  of 
the  transfer  from  the  first  specific  context  to  the  general  context  (at  90%)  and  the 
transfer  from  the  general  context  to  the  second  specific  context  (at  90%),  for  81% 
transfer  between  tasks. 

The  assumptions  described  in  the  previous  paragraph  are  used  to  calculate 
the  parameters  of  the  task  training  function  in  the  following  manner.  The  81%  of 
the  ekill  that  is  common  to  all  specific  contexts  will  be  learned  once  when  the  first 
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task  is  trained.  The  remaining  19%  of  the  skill  is  specific  to  the  task  contexts;  it 
will  be  learned  separately  for  the  initial  task  and  each  of  the  remaining  four 
tasks.  Thus,  the  total  amount  of  general  and  specific  skill  to  be  learned  is  0.81  + 
(5  x  0.19)  ss  1.76.  This  is  only  one  way  to  estimate  the  amount  to  be  trained,  and 
is  a  conservative  estimate.  We  set  the  learning  rate,  C,  to  reflect  the  fact  there  is 
more  to  be  learned  in  task  training  than  in  skill  training.  Since  the  value  of  C  for 
skill  training  is  1.5,  we  may  adjust  C  to  equal  1.6  +  1.76  =  0.852.  Since  task 
training  addresses  the  skill  in  all  relevant  contexts,  we  set  A  =  1.0.  Other 
parameters  are  the  same  as  for  skill  training  (S  =  0.0,  k  =  0.7). 

The  learning  and  cost-effectiveness  functions  that  are  derived  from  these 
assumptions  are  shown  in  Figure  7.  The  performance  levels  indicated  for  task 
training  in  the  two  graphs  represent  the  arithmetic  average  performance  level 
over  the  common  portion  of  the  five  tasks.  The  results  show  a  small  difference 
between  the  learning  functions  representing  skill  training  and  task  training. 
However,  skill  training  has  a  cost-effectiveness  advantage  up  to  a  performance 
level  of  about  0.6.  At  the  minimal  performance  level,  skill  training  has  a  58%  cost 


effectiveness  advantage  over  task  training. 


Figure  7.  Multiple-task  learning  and  cost-effectiveness  functions  assuming  no 
time  compression  or  reduced  training  cost. 
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The  effect  of  generalization  for  this  example  was  considerably  less  than  the 
improvement  that  was  obtained  through  time  compression.  However,  the  impact 
of  skill  generalization  depends  on  the  number  of  tasks  to  which  skills  will 
generalize,  the  skill-to-task  transfer  rate,  and  the  task-to-task  transfer  rate.  We 
assumed  that  both  of  the  transfer  rates  were  fairly  high.  If  the  task-to-task 
transfer  rate  were  reduced  from  the  assumed  81%  to  0%,  leaving  the  skill-to-task 
transfer  rate  at  90%,  then  the  relative  advantage  of  skill  training  would  increase 
from  68%  to  450%;  if  the  rate  were  increased  from  81%  to  100%,  then  the 
advantage  of  skill  training  would  be  eliminated  and  task  training  would  have  a 
10%  advantage.  A  similar  result  occurs  when  skill-to-task  transfer  is  changed.  If 
this  rate  is  reduced  from  its  assumed  90%  to  below  67%,  leaving  the  task-to-task 
transfer  rate  at  81%,  then  there  is  no  advantage  to  skill  training  at  any 
performance  level. 

The  preceding  analysis  indicates  some  of  the  conditions  that  should  occur 
for  skill  training  to  produce  cost-effective  training  because  of  6kill  generalization. 

1.  The  effect  of  skill  generalization  is  enhanced  if  the  initial  performance  level 
is  low.  As  was  the  case  for  time  compression,  the  maximum  advantage  for 
skill  training  occurs  at  low  performance  levels. 

2.  The  effect  of  skill  generalization  is  enhanced  when  a  skill  generalizes  to  a 
large  number  of  tasks. 

3.  Effective  skill  generalization  requires  that  the  transfer  of  training  from  skill 
to  task  is  high.  Low  transfer  of  training  has  the  double  effects  of  lowering 
the  initial  cost-effectiveness  (at  performance  level  0.0)  and  increasing  the 
rate  at  which  cost-effectiveness  decreases  as  a  function  of  performance  level. 
High  skill-to-task  transfer  indicates  that  the  context  plays  little  role  in  the 
various  instantiations  of  the  skill.  Hence,  we  would  expect  higher  skill-to- 
task  transfer  for  skills  in  which  there  is  a  consistent  mapping  between 
stimuli  and  responses. 

4.  The  effect  of  skill  generalization  is  enhanced  if  transfer  from  task  to  task  is 
low.  If  transfer  between  tasks  is  sufficiently  high,  tasks  may  generalize 
directly  as  well  as  skills  generalize  to  tasks.  Hence,  there  would  be  little 
gained  from  skill  training.  We  expect  that  tasks  requiring  cognitive  skills, 
such  as  planning,  would  show  low  task-to-task  transfer  because  the  same 
planning  heuristic  often  appears  different  in  different  situations. 

Reduction  of  Attention  Requirements 

The  final  benefit  of  skill  training  to  be  discussed  is  its  ability  to  alleviate 
the  attentional  demands  in  the  early  stages  of  training  of  complex  tasks. 

To  illustrate  the  benefits  of  skill  training  for  complex  tasks,  we  will  use  the 
learning  function  that  represent  the  effects  of  attentional  limitations  on  learning. 
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We  will  assume  that  initially  the  task  requires  three  times  the  attentional 
resources  that  the  student  has  available  (i.e.,  R  =  3,  L  =  1  for  task  training).  As 
the  task  is  learned,  the  resources  required  gradually  reduce,  until  at  the 
performance  level  0.67,  the  requirements  are  within  the  capabilities  of  the 
student.  At  that  performance  level,  learning  proceeds  at  its  nominal  rate. 

Otherwise,  our  assumptions  will  be  the  same  as  were  used  in  the  single¬ 
task  functions  described  in  Figure  6.  Specifically,  for  task  training,  we  assume 
that  C  =  0.5,  S  =  0.0,  k  =  0.7,  and  A  =  1.0.  For  skill  training,  we  assume  that 
C  =  1.5,  S  =  0.0,  k  =  0.7,  and  A  =  0.3.  Furthermore,  for  skill  training  we  assume 
that  L  >  R.  That  is,  the  skill  does  not  require  so  much  attention  to  hinder 
learning. 

Figure  8  shows  the  learning  and  cost-effectiveness  functions  for  skill 
training  and  task  training.  To  examine  the  effects  of  limited  attention,  this  figure 
should  be  compared  to  Figure  6,  which  has  the  identical  situation  without  any 
effects  of  limited  attention.  The  skill  training  curves  are  the  same  in  the  two 
figures.  However,  the  task  training  curves  are  noticeably  different.  In  Figure  8, 
early  task  learning  is  considerably  slower,  because  the  attentional  requirements  of 
the  task  exceed  the  capacity  of  the  student.  Thus,  skill  training  is  cost-effective, 
even  when  a  single  task  is  considered,  and  there  is  no  time  compression  or 
reduced  training  cost. 

The  advantage  of  skill  training  is  fairly  robust  to  changes  in  the  asymptote 
of  the  skill  learning  curve  or  the  task  attention  requirements.  The  asymptote 
must  be  lowered  to  0.11  for  the  advantage  of  skill  training  to  disappear. 

Similarly,  the  basic  result  remains,  albeit  at  a  reduced  level,  if  the  attention 
requirements  are  reduced  to  twice  the  student’s  attention  capacity.  It  should  be 
noted  that  the  analysis  does  not  recommend  training  the  skill  to  automaticity. 
Rather,  the  skill  is  trained  to  roughly  one-half  of  its  asymptotic  performance  level. 

Some  of  the  conditions  that  are  required  to  maximize  the  advantages  of  skill 
training  to  reduce  attention  requirements  are  enumerated  below. 

1.  The  major  requirement  is  that  the  task  is  sufficiently  complex  to  exceed  the 
attentional  capacity  of  the  student  during  early  learning.  Such  a  task  will 
require  a  long  time  to  train.  Early  learning  will  be  especially  slow,  and 
there  may  be  deviations  from  the  power  law  of  learning  at  low  performance 
levels. 

2.  The  advantages  of  skill  training  are  enhanced  if  the  skill  is  critical  to  the 
task,  unless  the  skill  itself  requires  greater  capacity  than  is  possessed  by 
the  student.  There  may  still  be  an  advantage  to  6kill  training  when  the 
skill  attention  requirements  exceed  the  capacity  of  the  student,  but  the 
advantage  will  be  reduced  because  the  skill  is  learned  at  a  slower  rate. 
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Figure  8.  Learning  and  cost-effectiveness  functions  illustrating  effects  of 
attentional  limits. 


3.  The  advantages  of  skill  training  are  greatest  when  the  initial  performance 
of  the  task  is  low.  When  the  performance  is  low,  the  task  requires  the 
greatest  resources  to  learn.  Hence,  there  is  the  greatest  advantage  of  skill 
training. 

4.  Reductions  in  attentional  requirement  seem  to  occur  only  for  skills  that 
involve  consistent  mapping  between  stimuli  and  responses  (Shiffrin  and 
Schneider,  1977;  Schneider  and  Shiffrin,  1977).  It  is  in  these  skills  in  which 
automatic  processing  is  possible. 
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EXAMPLE  OF  SKILL-BASED  TRAINING  DESIGN  MODEL 


In  our  characterization  of  skills  we  outlined  an  approach  for  identifying 
skills  for  training  outside  of  the  job  context,  based  on  a  body  of  research  from 
educational  and  cognitive  psychology.  The  rationale  presented  in  the  previous 
section  described  a  theoretical  framework  for  assessing  the  cost-effectiveness  of 
skill  training.  Neither  of  these  sections  specified  how  our  theoretical  approach 
may  be  translated  into  practical  application.  Before  describing  the  details  of  the 
skill-based  optimization  of  training  model,  it  would  be  a  valuable  exercise  to 
provide  a  concrete  example  of  the  steps  derived  from  that  theoretical  approach  in 
the  context  of  a  real  job  domain. 

The  model  breaks  the  specification  of  training  strategies  and  devices  into 
four  steps:  identifying  skills,  selecting  instructional  strategies,  designing  devices, 
and  allocating  training.  The  primary  objective  of  the  first  step  is  to  decompose 
tasks  into  their  elements  (i,e.,  performing  actions  upon  objects),  to  identify  the 
general  abilities  that  enable  the  performer  to  accomplish  the  task,  and  to  describe 
the  domain-specific  skills  that  will  become  candidates  for  skill-based  training.  The 
main  goal  of  the  second  step  is  to  group  and  sequence  skills  for  training.  In  the 
third  step,  device  requirements  are  derived  from  the  nature  of  the  skills  to  be 
taught.  These  requirements  include  instructional  capabilities,  interface  features, 
and  memory  requirements.  The  final  activity  compares  the  projected  cost  of  task 
training  to  that  of  task  training  supplemented  by  the  proposed  skill  training.  The 
example  will  focus  on  the  major  aspects  of  the  first  two  steps.  The  third  and 
fourth  steps  will  be  described  in  less  detail,  as  a  complete  description  of  these 
activities  would  require  a  more  thorough  description  of  the  first  two  activities  than 
is  practical  for  an  example. 

Description  of  the  Job  Domain 

The  example  we  use  is  from  the  Air  Traffic  Control  (ATC)  domain.  We  have 
chosen  ATC  to  use  as  an  example  for  several  reasons.  First,  we  wanted  to 
demonstrate  the  value  of  skill-based  training  in  a  domain  that  has  direct 
relevance  to  the  Army.  Second,  we  wanted  a  domain  with  which  we  have  some 
familiarity.  ATC  is  a  job  that  we  have  examined  in  some  detail,  and  we  can, 
therefore,  work  the  example  without  the  assistance  of  subject-matter  experts. 
Third,  ATC  requires  skills  corresponding  to  each  ability  category  (i.e.,  cognitive, 
perceptual,  psycnomotor).  ATC  is  a  highly  complex  real-time  job  that  demands 
planning  and  decision-making  skills  as  well  as  finely  tuned  perceptual  and  motor 
skills.  This  fact  is  important  since  the  training  research  and  development 
disciplines  are  just  beginning  to  recognize  the  role  of  cognitive  skills  in 
performance.  Consequently,  cognitive  skill  training  is  becoming  a  more  central 
component  of  instruction.  Finally,  the  domain  has  a  clear  task  structure  and  a 
sophisticated  environment  that  allows  issues  of  fidelity  and  instructional  context 
to  be  explored. 
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The  example  uses  as  a  point  of  departure  a  task  analysis  performed  for  the 
Federal  Aviation  Administration  on  the  job  of  en  route  ATC  (DOT/FAA/AP-87-Ol 
(Vol#6),  Ammerman,  Bergen,  Davies,  Hostetler,  Inman,  &  Jones,  1987).  The  FAA 
analysis  was  used  because  an  investigation  of  the  ATC  course  as  taught  in  the  ■ 
Army  (MOS  93C)  revealed  that  many  of  the  tasks  were  referenced  to  FAA 
standards.  The  task  analysis  was  conducted  to  provide  input  to  the  development 
of  new  ATC  equipment,  procedures,  and  instruction.  The  analysis  identified  348 
controller  tasks.  Each  task  is  defined  in  terms  of  information  (input/output) 
requirements,  frequency  and  criticality  of  the  task,  cognitive  and  sensory 
attributes  (general  ability  categories),  and  performance  criteria  (time  and 
accuracy).  For  the  most  part,  the  cognitive  and  sensory  attribute  list  shown  in 
Table  2  is  a  subset  of  our  general  ability  taxonomy  (Table  1).  Since  our  taxonomy 
is  a  more  complete  list  we  will  draw  upon  it  in  the  example. 

Tasks  are  specified  at  the  level  of  what  needs  to  get  done  (the  goal)  rather 
than  how  it  gets  done.  At  this  level  of  description,  the  skills  required  to  perform 
the  task  cannot  be  determined.  To  identify  how  tasks  are  accomplished  and, 
therefore,  the  skills  required,  tasks  were  decomposed  into  task  elements 
describing  the  procedural  steps  and  actions  used  in  accomplishing  the  task.  Task 
elements  are  statements  containing  a  verb,  which  specifies  some  action,  and  one 
or  more  nouns  representing  the  object  on  which  the  action  is  taken;  modifiers  are 
included  for  clarity.  The  list  of  task  element  verbs  is  organized  into  a  hierarchical 
verb  taxonomy  (Figure  9),  and  each  verb  is  defined  in  a  task  element  verb  glossary 
(see  Ammerman,  et  al.,  1987). 

Identify  Skills  for  Training 

Since  tasks  have  been  characterized  in  terms  of  the  abilities  required  to 
perform  them  and  domain-specific  actions,  objects,  procedures,  and  input/output 
information,  it  should  be  possible  to  derive  the  job  skills  associated  with  various 
tasks  in  a  systematic  way.  In  addition,  frequency  and  criticality  ratings  obtained 
for  each  task  and  the  recurrence  of  task  element  actions  across  tasks  should 
provide  a  way  of  selecting  those  skills  most  appropriate  for  skill-based  training. 

In  some  cases,  identifying  job  skills  will  be  straightforward  due  to  the  close 
correspondence  of  task  element  verbs  to  items  in  the  ability  taxonomy  (e.g.,  the 
verb  "scan"  and  the  ability  spatial  scanning).  In  other  cases,  it  will  be  more 
difficult  to  match  verbs  with  abilities  (e.g.,  the  verb  "project"  can  be  accomplished 
by  visualization  or  through  reasoning  processes).  The  source  of  the  identification 
problem  is  that  a  task  element  statement  can  use  a  verb  from  any  level  in  the 
hierarchy.  In  the  glossary,  however,  higher  level  verbs  are  defined  only  in  terms 
of  their  descendants.  For  example,  to  acouire  is  defined  as  perceive  via  detection, 
scanning,  search,  extraction,  or  cross-reference.  Since  each  of  these  low-level  verbs 
can  correspond  to  a  different  kind  of  cognitive  or  perceptual  ability,  it  would  be 
impossible  to  match  a  task  element  statement  containing  the  verb  acquire  with  a 
particular  skill.  In  these  ambiguous  situations,  more  detailed  knowledge  of  the 
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Table  2 

List  of  Cognitive/Sensory  Attributes  (from  DOT/FAA/AP-87-Ol  (Vol#l)) 


Coding 

Decoding 

Deductive 

Reasoning 

Filtering 


Image/Pattem 

Recognition 

Inductive 

Reasoning 


Long-Term 

Memory 

Mathematical/ 

Probabilistic 

Reasoning 

Movement 

Detection 

Prioritizing 

Short-Term 

Memory 

Spatial 

Scanning 

Verbal  Filtering 

Visualization 


Transformation  or  translation  of  information  for  entry  into  the  . 
system;  converting  textual  information  to  graphics  or  symbols. 

Transformation  or  translation  of  information  received. 

Ability  to  reach  a  conclusion  that  follows  logically  from  the 
known  facts  or  data;  selection  from  among  alternative  answers 
or  methods. 

Selection  of  inputs  on  which  to  focus  attention  in  the  presence  of 
distracting  stimuli  or  high  workload;  selective  attention; 
overload  accommodation. 

Perception  of  spatial  patterns  and  relations  among  static  or 
dynamic  visual  inputs.  May  involve  orienting  oneself  to  the 
position  or  configuration. 

Generation  of  an  explanation  for  a  set  of  specific  data  or 
instances,  giving  structure  and  meaning  to  the  information; 
Generalization  of  working  hypotheses  from  specific  events; ... 
ability  to  make  a  knowledgeable  assumption  using  incomplete 
data. 

Mental  storage  of  knowledge  over  a  period  of  time  and 
selective  recall  of  items  relevant  to  the  situation. 

Translation  of  uncertainty  into  probability;  assigning  a 
subjective  probability  regarding  the  likelihood  of  an  event 
occurring,  ability  to  use  probabilities  to  identify  optimal  courses 
of  action. 

Recognition  of  the  physical  movement  of  a  visual  object; 
estimation  of  its  direction  or  speed. 

Ordering  of  events  in  sequence;  establishing  priorities. 

Mental  storage  and  selective  recall  of  relevant  information 
over  a  brief  period  of  time. 

Rapid  identification  or  detection  of  objects  or  events  displayed  in 
a  wide  or  complicated  visual  field. 

Same  as  Filtering,  but  limited  to  voice  communications. 

Observation  of  spatial  patterns  and  subsequent  mental 
transformations  into  other  spatial  patterns. 


USER-INTERNAL  TAXONOMY 


DETECT 

SEARCH 

ACQUIRE  SCAN 


USER-INPUT  TAXONOMY 


NAME 

GROUP 


PERCEIVE 


EXTRACT 

CROSS-REFERENCE 


^identify  discriminate 

*  RECOGNIZE 


ANALYZE 


CATEGORIZE 

CALCULATE 

ITEMIZE 

TABULATE 


MEDIATE 


ESTIMATE 
INTERPOLATE 
TRANSLATE 
SYNTHESIZE  INTEGRATE 
FORMULATE 
PROJECT/ 
EXTRAPOLATE 


ASSESS 


DECIDE 


TRANSMIT 


COMMUNICATE 


COMPARE 

EVALUATE 


CALL 

ACKNOWLEDGE 

RESPOND 

SUGGEST 

DIRECT 

INFORM 

INSTRUCT 

REQUEST 


RECEIVE 


CREATE 

ASSOCIATE 

INTRODUCE 

ASSEMBLE 

REPLICATE 

INDICATE 

INITIATE 

REFERENCE 

ELIMINATE 

REMOVE 

STOP 

DISASSOCIATE 

DISASSEMBLE 

SUPPRESS 

SET-ASIDE 

MANIPULATE 

TRANSFORM 

(CHANGE 

ATTRIBUTE) 

ACTIVATE 

PERFORM 
< _ TEM) 

EXECUTE 
( _ FUNCTION) 

INSERT 


AGGREGATE 

OVERLAY 


COPY 

INSTANCE 


CUT 

DELETE 


SUSPEND 

TERMINATE 


RENAME 

UNGROUP 


SEGREGATE 

FILTER 


Figure  9.  Task  element  verb  taxonomy  (from  DOT/FAA/AP-87-Ol  (Vol  #1)). 
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job  domain  is  needed  to  infer  the  proper  job  skill.  This  knowledge  can  be  supplied 
by  subject-matter  experts  or  through  supplemental  job  analysis. 

Identify  Tasks 

Tasks  are  selected  for  consideration  based  on  three  criteria:  entry-level 
performance,  training  time,  and  workload.  These  constructs  are  described  in  our 
formal  model  description  and  will  not  be  discussed  further  here.  For  the  example, 
we  have  selected  one  of  the  en  route  radar  controller’s  major  responsibilities, 
ensuring  that  aircraft  are  separated  from  each  other  and  airspace  boundaries 
according  to  prescribed  separation  standards.  Part  of  this  responsibility  is 
superordinate  Task  Al.1.1:  Checking  and  Evaluating  Separation  (Ammerman,  et 
al.,  1987).  The  purpose  of  Task  Al.1.1  is  to  anticipate  potential  conflicts  between 
aircraft  before  separation  is  lost.  In  general,  this  is  done  by  monitoring  radar 
tracks  on  the  Plan  View  Display  (PVD),  or  radar  scope,  and/or  by  gathering  and 
interpreting  data  from  several  sources  (i.e,  data  blocks,  flight  progress  strips) 
concerning  the  route  of  flight,  altitude,  speed,  and  estimated  time  at  various 
intervals  for  each  aircraft  in  (or  about  to  enter)  the  sector. 

Task  Al.1.1  involves  fifteen  subordinate  tasks  and  a  total  of  116  task 
elements.  Three  of  those  tasks  are  listed  in  Table  3.  We  will  use  this  set  of  tasks 
to  illustrate  the  skill  identification  process.  Many  of  the  task  elements  are  highly 
similar  within  and  across  these  subordinate  tasks.  In  fact,  across  the  116 
elements  there  are  only  13  unique  verbs.  The  unique  task  element  verbs 
associated  with  each  task  in  Table  3  are  listed  beneath  each  task  statement.  Also 
listed  are  the  cognitive/sensory  attributes  associated  with  the  task.  Note  that 
although  the  verb  lists  are  very  similar  across  these  tasks,  the  attribute  lists  vary 
significantly. 

Identify  General  Abilities 

We  begin  the  process  by  reviewing  each  task  element,  attempting  to  identify 
the  general  abilities  characterized  by  the  task  element  verb.  The  first  step  i6  to 
identify  the  abilities  in  the  ability  taxonomy  (Table  1)  that  might  be  involved  in 
performing  the  action  specified  by  the  task  element  verb.  While  it  may  seem 
possible  to  simply  match  verbs  in  the  verb  taxonomy  to  abilities  in  the  general 
ability  taxonomy,  in  practice  there  is  no  straightforward  mapping  scheme.  The 
mapping  often  depends  upon  the  context  of  the  particular  task.  Identifying  the 
appropriate  ability  can  require  domain-specific  knowledge  that  is  not  supplied  by 
the  verbs  alone. 

Search.  The  first  verb  in  Table  3  is  "search,"  which  is  defined  as 
"purposeful  looking  over  a  display  or  area  to  locate  a  specific  item  or  items."  An 
example  of  a  task  element  containing  this  verb  is  Al. 1.1.30. 2: 

Search  flight  progress  strip  in  flight  strip  bay  for  information 

pertaining  to  aircraft  separation. 
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Table  3 

Partial  Task  List  for  Al.1.1,  Checking  and  Evaluating  Separation 


Al. 1.1.2  Review  plan  view  display  for  potential  violation  of  aircraft 
separation  standards. 


Task  Element  Verbs: 

search 

perceive 

extract 

synthesize 

recognize 

Al. 1.1.4  Project  mentally  an  ai 

Task  Element  Verbs: 

search 

extract 

synthesize 

project 


Cognitive/Sensory  Attributes: 

movement  detection 
spatial  scanning 
filtering 
visualization 
inductive  reasoning 
math/prob  reasoning 

aft’s  future  position/altitude/path. 

Cognitive/Sensory  Attributes: 

visualization 
short-term  memory 
inductive  reasoning 


Al.  1.1.30  Review  flight  progress  strips  for  present  and/or  future  aircraft 
separation. 


Task  Element  Verbs: 

search 

extract 

synthesize 

recognize 


Cognitive/Sensory  Attributes: 

spatial  scanning 
decoding 
visualization 
short-term  memory 
inductive  reasoning 
math/prob  reasoning 
prioritizing 


Searching  for  a  specific  item  can  involve  several  abilities,  including 
detection,  pattern  recognition,  selective  attention  (filtering),  and/or  discrimination. 
We  must  a6k  which  ability  or  abilities  apply  in  the  context  of  the  particular  task 
element.  Detection  can  be  eliminated  because  skill  acquisition  in  detection  is  only 
an  issue  when  stimuli  are  difficult  to  detect.  The  objects  to  be  detected  in  this 
case,  flight  strip  data  fields,  are  easily  discemable.  Discrimination  requires  two  or 
more  objects  to  discriminate  and  a  list  of  attributes  on  which  the  discrimination  is 
based.  This  task  element  does  not  have  these  components.  Selective  attention  is 
generally  associated  with  vigilance  monitoring  tasks,  in  which  a  target  stimulus 
must  be  detected  against  a  background  of  competing  stimuli.  Again,  this  is  not 
the  case.  The  only  ability  that  would  reliably  distinguish  among  levels  of 
performance  is  pattern  recognition. 
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The  pattern  is  defined  by  the  object(s)  in  the  environment  associated  with 
the  task  element,  the  area  of  regard,  and  the  information  processing  modality. 

For  this  task  element,  the  objects  are  alphanumeric  flight  strip  data  fields,  the 
area  of  regard  is  the  flight  strip  bay,  and  the  modality  is  visual.  The  result  is  the 
location  of  a  specific  data  field  or  fields. 

Having  gone  through  this  process,  we  can  generalize  that  any  "search"  task 
element  having  the  same  class  of  objects,  area  of  regard,  and  modality  will  require 
the  same  general  ability.  An  example  would  be  task  element  Al.  1.1.31.1: 

Search  flight  progress  strip  in  flight  strip  bay  for  information 
pertaining  to  potential  violation  of  flow  restrictions. 

However,  all  tasks  elements  containing  a  particular  verb  may  not  invoke  the  same 
set  of  abilities.  It  is  important  to  distinguish  among  classes  of  objects  to  which  the 
task  action  is  applied  when  identifying  abilities.  "Search"  task  elements  can 
involve  abilities  other  than  alphanumeric  pattern  recognition  as  in  the  following 
example,  Al.l. 1.2.1: 

Search  primary  target,  data  block  for  potential  violation  of  aircraft 
separation  standards. 

Primary  targets,  or  blips,  can  appear  anywhere  on  the  radar  display  at  any  time 
among  a  great  deal  of  "clutter."  Consequently,  detection  becomes  a  factor  in  the 
search  for  targets.  This  is  not  true  for  data  blocks,  which  like  flight  strips  are 
readily  discemable. 

Perceive.  The  next  task  element  verb  on  the  list  (under  Al.l. 1.2,  Table  3)  is 
"perceive."  Unfortunately,  the  three  top-level  verbs  in  the  task-element 
taxonomy — perceive,  mediate,  and  communicate — are  not  defined  in  the  task- 
element  verb  glossary;  they  are  only  category  titles.  The  task  element  in  which  it 
i6  used  is  Al.l.  1.2. 2.1: 

Perceive  plan  view  mental  traffic  picture  from  target  position  symbol, 
track  status  symbol,  track  history,  velocity  vector  on  plan  view 
display. 

The  fact  that  a  top-level  verb  is  used  in  a  task  element  reveals  that  the  mental 
processes  involved  are  not  well-understood.  Any  or  all  of  the  lower-level 
perceptual  verbs  may  be  involved.  Based  on  an  understanding  of  the  objects  in 
this  task  element  and  the  kinds  of  information  they  convey  (e.g.,  controllers  can 
determine  direction  of  flight  and  derive  speed  estimates  from  track  history)  we  can 
reason  that  this  task  involves  quantitative  estimation,  inductive  reasoning,  spatial 
reasoning,  and/or  visualization.  We  would  require  more  information  to  make  a 
definite  determination,  and  defer  any  recommendations  until  more  detailed 
analysis  is  conducted. 
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Extract.  The  next  verb  from  Table  3,  "extract,"  is  defined  by  Ammerman,  et 
al.  (1987)  as  "directed,  attentive  reading,  observing,  or  listening  with  the  purpose 
of  gleaning  the  meaning  or  contents  thereof."  A  sample  task  element  statement  is 
Al. 1.1.30.3.3: 

Extract  route  information,  previous  posted  fix,  posted  fix,  next  posted 

fix  from  flight  progress  strip. 

In  this  data  set,  extraction  is  generally  associated  with  search.  It  encodes 
the  meaning  of  an  item  in  active  memory  and  serves  to  complete  the  search. 
Extraction  of  meaning  can  be  indicative  of  verbal  comprehension  or  perceptual 
pattern  recognition  ability.  By  definition,  it  can  involve  either  visual  or  auditory 
modality.  In  the  sample  statement,  however,  the  objects  are  alphanumeric  data  of 
specific  values  found  in  flight  data  fields  and  identified  by  the  visual  search 
process.  Therefore,  the  extraction  of  meaning  is  more  appropriately  associated 
with  the  internal  (mental)  modality. 

Synthesize.  The  verb  "synthesize"  is  a  mid-level  verb  that  is  defined  by 
Ammerman,  et  al.  (1987)  as  mentally  producing  new  information  via  estimation, 
interpolation,  translation,  integration,  formulation,  or  projection  (its  subordinate 
verbs).  Again,  it  is  clear  that  this  definition  leaves  room  for  several  kinds  of 
skills.  To  determine  which  are  appropriate  we  must  examine  specific  task  element 
statements.  One  such  statement  is  Al. 1.1.30.4: 

Synthesize  position,  route,  speed,  altitude,  and  time  information  into 

a  mental  picture  of  aircraft  separation. 

In  this  statement,  the  objects  are  alphanumeric  information  extracted  from 
flight  strip  data  fields.  To  determine  the  modality  it  is  necessary  to  look  at  the 
context  of  prior  task  element  statements.  Since  those  statements  are  "extracts," 
we  can  assume  that  the  information  resides  in  active  memory;  thus  the  modality 
is  internal.  Given  that  the  product  of  the  synthesis  is  a  mental  picture,  we  can 
assume  that  at  a  minimum,  translation  (changing  from  one  representational 
system  to  another)  and  integration  (mentally  organizing  a  variety  of  data)  are 
involved.  The  general  abilities  that  underlie  these  processes  are  abstraction  and 
6patial  reasoning.  If,  indeed,  an  analog  mental  picture  is  formed  then 
visualization  would  also  be  a  candidate. 

Recognize.  The  next  verb  from  Table  3,  "recognize,"  is  defined  by 
Ammerman  et  al.  (1987)  as  "specific,  positive  identification  of  an  entity."  A 
statement  containing  this  verb  is  Al.l. 1.2.4: 

Recognize  potential  violation  of  separation  standards. 

This  is  another  instance  in  which  context  is  important.  On  the  surface,  this 
verb  is  indicative  of  pattern  recognition  ability.  As  we  found  in  the  case  of 
"search"  and  "extract,"  however,  pattern  recognition  can  refer  to  different  classes 
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of  objects  across  modalities.  In  this  case,  the  target  pattern,  or  object,  is  a  6patial 
mapping  of  aircraft  in  an  internal  (mental)  picture  of  the  airspace.  Also  implicit 
in  the  statement  is  the  ability  to  encode  and  recall  separation  standards,  or  rules, 
stored  in  long-term  memory.  Hence,  this  verb  also  characterizes  the 
interpretation  of  a  data  pattern  according  to  established  rules,  or  deduction. 

Identify  Skills 

Based  on  this  analysis  of  task  element  statements  we  have  identified 
several  general  abilities  that  underlie  performance  on  Task  Al.1.1,  the  conditions 
under  which  those  abilities  are  invoked,  and  the  environment  in  which  they  are 
applied.  We  have  discovered  that  at  the  level  of  detail  specified  in  the  FAA 
analysis,  tasks  and  even  task  elements  do  not  unambiguously  distinguish  among 
skills  (i.e.,  there  exists  no  one-to-one  mapping).  Having  identified  the  general 
abilities,  we  can  now  trace  back  and  translate  this  information  into  job  skills. 

Keeping  in  mind  our  goal  of  developing  skill-based  training  formats,  we 
would  not  want  to  define  skills  so  specifically  as  to  distinguish  among  all  task 
elements.  Rather,  we  want  to  identify  skills  that  apply  across  task  elements.  To 
accomplish  this,  we  have  created  a  skill  frame  hierarchy: 

verb  (modality) 
ability 

area  of  regard 

task  object  category:  objects 

A  skill  is  defined  as  the  application  of  an  ability  to  a  class  of  objects  in  an 
area  of  regard.  When  a  new  task  element  is  encountered,  the  information 
contained  in  it  can  be  compared  to  existing  data  and,  if  necessary,  added  to  the 
hierarchy.  We  acknowledge  the  bottom-up  nature  of  the  process;  development  of 
the  frame  structure  is  driven  by  the  available  data.  The  more  data  sets  the  model 
is  applied  to,  the  more  complete  the  frame  hierarchy  will  become. 

Below  are  the  frames  identified  for  the  task  action  verbs  search,  extract, 
synthesize,  and  recognize. 

Search  (visual) 

pattern  recognition 
radar  display 

graphic:  track  history,  velocity  vector 
field:  data  block 
text:  flight  data 
flight  strip  bay 

field:  flight  data  fields 
text:  flight  data 
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detection 

radar  display 

graphic:  primary  target 

Some  skills  correspond  to  information  gathering  activities.  We  have 
identified  two  such  skills  associated  with  the  task  element  search:  detecting 
primary  targets  on  the  radar  display  and  recognizing  various  kinds  of  consistently 
mapped  patterns.  For  pattern  recognition,  we  identified  two  areas  of  regard:  the 
flight  strip  bay  and  the  radar  display.  Objects  within  these  two  areas  were  of 
three  types:  graphic  symbols,  text,  and  data  fields.  The  modality  was  visual.  We 
can,  therefore,  conclude  that  an  important  job  skill  is  having  some  proficiency  at 
identifying  those  objects  in  their  respective  contexts.  For  certain  graphic  symbols 
(i.e.,  primary  targets)  it  is  also  important  to  develop  skill  at  detecting  the  symbols 
against  a  background  of  competing  symbols  in  variable  locations. 

Extract  (visual) 

verbal  comprehension 

flight  strip  data  field 

text:  route  info,  previous  posted  fix,  posted  fix, 
next  posted  fix 

Another  important  information  gathering  skill  is  the  ability  to  comprehend 
the  meaning  of  information.  For  example,  the  numeral  220  in  the  flight  progress 
strip  altitude  data  field  means  that  the  aircraft  is  flying  at  22,000  feet.  In  another 
field  it  may  mean  a  heading  of  220  degrees.  Controllers  must  be  able  to  interpret 
information  in  the  surrounding  context. 

Synthesize  (internal) 
abstraction 

text:  route,  speed,  altitude,  time 
spatial  reasoning 

mental  representation:  aircraft  relationships 

visualization 

mental  image:  analog  picture  of  airspace 

Recognize  (internal) 

information  recall 

rules:  separation  standards 

deduction 

data:  actual  separation,  min  legal  separation 

At  a  higher  level,  controllers  must  have  information  manipulation  skills. 

We  have  identified  the  ability  to  estimate  distance  based  upon  speed  and  time 
data  and  to  estimate  speed  based  on  history  markings.  These  skills,  in  turn,  allow 
controllers  to  make  spatial  judgments  and  comparisons,  such  as  determining  the 
intersection  of  two  flight  paths  or  whether  an  established  rule  has  been  violated. 
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Skill  Training  Strategies 


We  have  argued  that  skill-based  training  is  intended  to  complement  task- 
based  training.  After  identifying  a  set  of  job  skills,  the  next  step  is  to  determine 
which  are  appropriate  for  skill-based  training,  and  which  are  more  appropriately 
taught  in  a  task  context.  Some  of  the  factors  that  should  be  considered  are: 

•  level  of  training  effort 

•  consistency  across  instances 

•  level  of  contextual  dependence 

•  frequency  of  use  on  the  job 

•  criticality  to  job  performance 

•  level  of  proficiency  required  by  the  job 

When  a  skill  is  narrowly  defined  or  requires  few  mental  resources  to 
acquire,  it  may  be  more  cost-effective  to  incorporate  its  instruction  into  task-based 
lessons.  An  example  might  be  the  ability  to  read  time  from  the  military  clock  on 
the  radar  display.  Skills  whose  application  are  highly  context-dependent  may  also 
be  better  candidates  for  a  job  task  environment.  Conversely,  skills  that  apply 
more  generally  across  tasks  and  those  that  require  much  effort  to  acquire  are 
better  taught,  at  least  initially,  in  a  skill-based  environment. 

Frequency  and  criticality  are  also  important  factors.  Generally,  the  more 
critical  the  skill  is  to  job  proficiency,  the  more  training  should  emphasize  the 
development  of  that  skill.  Similarly,  the  more  frequently  one  is  called  upon  to  use 
a  skill,  the  more  likely  initial  skill  training  will  enhance  learning.  Confounded 
with  these  factors  is  the  target  proficiency  level.  Are  errors  tolerated?  Is  speed  a 
factor?  Must  the  task  be  performed  concurrently  with  other  more  or  less 
demanding  tasks?  Skills  that  must  be  applied  without  sapping  attentional 
resources  (automatic  processes)  require  many  more  practice  trials  to  develop  than 
are  generally  given  in  task-based  training. 

Skill  training  strategies  specify  the  skills  that  should  be  trained  together, 
the  level  of  performance  to  which  skills  should  be  trained,  and  how  to  sequence 
training.  Grouping  skills  for  training  is  based  upon  category  similarity  in  the  skill 
frame  hierarchy.  For  example,  recognizing  data  fields  on  flight  strips  might  be 
grouped  with  comprehension  of  the  text  in  each  field.  This  might  be  followed  by 
training  on  data  block  fields.  Since  these  skills  are  applied  on  objects  which  are 
consistently  mapped  (e.g.,  220  in  the  altitude  field  always  means  22,000  feet)  the 
performance  criterion  would  be  high.  An  example  of  sequencing  would  be  training 
information  gathering  skills,  followed  by  training  on  information  manipulation 
skills  such  as  abstraction  of  the  data  into  a  mental  representation  of  aircraft 
positions,  and  training  on  spatial  reasoning  about  that  representation. 

A  device  used  to  provide  training  on  the  specified  skill  strategies  requires 
certain  interface  and  instructional  features  to  be  successful.  The  final  step  in  the 
development  of  the  skill  strategies  is  to  determine  the  instructor  support  and 
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interface  requirements  associated  with  each  skill  strategy.  These  two  kinds  of 
requirements  are  determined  through  the  use  of  rules  that  specify  requirements 
as  a  function  of  general  ability,  modality,  skill  objects,  training  criterion, 
sequencing  method,  student  entry  characteristics,  and  the  domain  knowledge  of 
the  training  designer  or  subject-matter  expert.  The  requirements  provide  the 
major  control  for  the  design  of  skill-training  devices  described  in  the  following 
section. 

Instructor  support  features  are  elements  of  training  devices  that  aid  the 
instructional  process,  producing  more  efficient  training.  Examples  of  instructor 
support  features  include  scenario  control,  performance  measurement,  augmented 
feedback,  augmented  cues,  and  performance  recording  and  playback.  Factors  to 
consider  are  primarily  the  basic  ability  required  by  the  skill,  student  skill  level, 
and  the  skill  training  criterion.  For  example  a  detection  task  can  benefit  from  the 
use  of  augmented  cues,  but  only  when  the  student  entry  skills  are  low.  Other 
instructor  support  features  are  appropriate  for  other  types  of  skills  or  when  the 
skill  training  criterion  is  high.  Thus,  there  is  a  set  of  rules  that  we  may  use  to 
determine  the  instructor  support  requirements  for  each  skill  strategy. 

Interface  requirements,  like  instructor  support  requirements,  are 
determined  by  a  set  of  rules  that  recommend  interface  features  based  on  skill 
information.  Most  interface  requirements  come  from  the  need  to  present  cues 
such  as  visual  displays  and  instrument  readings,  to  provide  controls  for  learner 
responses,  and  to  present  response  feedback.  Interface  requirements  are  based 
primarily  on  the  basic  ability,  modality,  skill  objects,  and  domain  information 
provided  by  subject-matter  experts.  For  example,  the  group  of  skills  involving 
recognition  of  the  data  fields  on  a  flight  strip  requires  a  text  display,  because  the 
fields  on  the  flight  strip  are  textual  information.  Training  these  skills  does  not 
require  a  simulation  capability  because  the  flight  strip  is  a  static  display. 

Design  Skill-Training  Devices 

The  previous  analysis  has  produced  training  strategies  that  include  groups 
of  skills  and  their  associated  instructor  support  and  interface  requirements.  A 
variety  of  training  devices  of  several  different  categories  could  be  used  to 
implement  the  training  strategies.  Some  of  the  categories  of  training  device  are 
well-defined  technologies  with  known  options.  Examples  of  these  general-purpose 
technologies  are  computer-based  instruction  (CBI),  panel  trainers,  and  control 
procedure  trainers.  These  trainer  categories  provide  general  capabilities  that  may 
be  adequate  to  implement  many  of  the  skill-training  strategies.  Other  strategies 
will  require  the  design  of  special-purpose  training  devices  or  simulators  tailored  to 
their  specific  device  requirements. 

The  procedure  for  choosing  a  design  of  a  training  device  from  a  general 
device  category  is  similar  in  many  respects  to  the  procedure  for  designing  a 
special-purpose  training  device.  In  each  case,  the  device  consists  of  several 
components  in  which  there  are  design  options.  These  design  options  reflect  the 
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sophistication  of  displays,  the  amount  of  interactive  control,  the  processing  power, 
the  peripheral  devices,  and  other  features.  The  design  options  vary  in  their  cost 
and  sophistication,  as  well  as  in  their  capability  to  meet  the  requirement  of  the 
skill  strategies.  To  be  effective  a  design  must  meet  the  interface  and  instructor 
support  requirements  of  the  skill  strategies  that  are  to  be  implemented  on  the 
device. 

The  major  difference  between  the  processes  of  choosing  the  design  of  a 
general-purpose  device  and  developing  the  design  for  a  special-purpose  device  is  in 
the  number  and  range  of  options  available.  General-purpose  training  devices  have 
a  limited  number  of  relatively  well-defined  design  options.  However,  the  range  of 
options  is  much  wider  for  special-purpose  devices.  In  each  case,  the  task  is  to 
select  the  design  that  meets  the  requirements  of  skill  strategies  at  the  minimum 
cost. 


The  procedure  we  describe  begins  by  analyzing  general-purpose  training 
devices  and  ordering  on  increasing  cost.  For  each  training-device  category,  the 
skill  groups  are  identified  that  could  be  trained  by  the  most  sophisticated  training 
device  in  that  category.  This  cluster  of  skill  groups  is  used  to  evaluate  the  device¬ 
category  interface  and  instructor  support  options.  Each  option  receives  benefit 
according  to  the  number  of  skill  groups  for  which  it  meets  or  exceeds  the 
requirements. 

As  an  example  of  the  design  skill-training  devices  activity  we  shall  consider 
the  design  of  a  CBI  system.  The  components  for  such  a  system  include  text  and 
graphic  visual  displays,  keyboard  and  pointing  input  devices,  moderate 
computation  and  simulation  capabilities,  and  possibly  access  to  video  images. 

After  sorting  through  the  skills  that  have  been  assigned  skill  training  strategies,  a 
list  of  skills  groups  for  this  type  of  device  might  contain  the  following: 

•  Recognition  of  data  fields  on  a  data  strip  and  in  data  block 

•  Comprehension  of  data  in  a  field 

•  Detection  of  targets  on  a  radar  display 

•  Abstraction  of  a  mental  representation  of  aircraft  data 

•  Spatial  reasoning  (e.g.,  projection  of  a  future  position  for  an  aircraft) 

The  second  activity  required  to  design  skill -training  devices  is  to  identify 
instructional  features,  and  to  develop  costs  and  benefits  for  them.  These  benefits 
and  costs  can  be  used  to  develop  benefit  to  cost  ratios,  which  can  be  used  to  rank 
and  select  features  addressing  the  skill  groups.  Exemplary  features  include  cue 
augmentation,  feedback,  and  monitoring.  The  requirements  for  instructional 
support  features  for  each  skill  group  are  examined,  and  each  instructional  support 
feature  can  receive  a  benefit  in  proportion  to  the  number  of  skill  groups  wrhich  the 
feature  addresses.  The  addressed  skill  groups  contribute  a  weight  to  this 
calculation  according  to  their  criticality  and  training  need.  Instructional  support 
features  can  then  be  selected  according  to  the  ratio  of  this  measure  of  benefit  to 
their  cost. 
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The  third  activity  selects  interface  features  by  device  and  produces  costs 
and  benefits  for  options  within  several  of  the  device’s  interface  components.  These 
assigned  benefits  and  costs  can  be  used  to  rank  and  select  the  interface  features, 
as  was  proposed  with  instructional  features.  The  category  of  device  being 
considered  limits  the  range  of  components  and  the  options  within  each  component. 
For  example,  display  size  is  an  interface  component  having  several  options, 
including  standard,  large  and  projection.  While  many  other  display  sizes  are 
possible,  these  three  options  represent  the  major  alternatives  possible  with 
computer-based  devices.  Display  characteristics,  another  component,  has  several 
options:  text  only,  text  or  graphics,  text  and  graphics  in  monochrome,  in  color, 
and  in  high  resolution  color.  Components  which  may  either  be  present  or  absent 
include  the  pointing  device  and  stored  video  images.  The  computation  capabilities 
could  be  designated  by  the  processor  type  (e.g.,  8088,  80286,  80386,  and  80486) 
and  speed  (e.g.,  8,  12,  16,  20,  or  33  MegaHertz).  Finally  the  options  for  simulation 
capability  could  be  none,  commercial  off-the-shelf  package,  a  CBI  system,  and 
specialized  software. 

The  benefit  of  an  option  for  a  component  can  be  determined  by  comparing 
the  capability  of  the  option  to  the  interface  requirements  for  each  skill  group.  In 
general,  there  will  be  a  direct  correspondence  between  device  components  and 
interface  dimensions.  The  benefit  of  any  option  can  be  the  weighted  number  of 
skill  groups  for  which  the  capability  of  the  option  is  greater  than  the  requirement 
of  the  skill  group  on  the  relevant  interface  dimension.  The  skill  groups  are 
weighted  by  their  criticality  and  training  need,  as  above.  The  index  of  merit  for 
each  component  option  can  be  the  incremental  benefit  divided  by  the  incremental 
cost  of  the  option.  The  result  of  this  activity  is  the  prioritized  set  of  device 
options,  based  on  a  cost-benefit  analysis. 

The  fourth  activity  determines  the  priorities  of  the  device  interface  options 
and  instructional  features  based  upon  their  relative  benefit-to-cost  ratios.  The 
designer  then  selects  the  device  design  based  upon  a  desired  cost  or  benefit  level 
to  be  achieved.  An  illustrative  device  design  for  $5000  and  75%  of  the  total  skill 
training  benefit  might  be: 

Text  and/or  graphics,  color 
Standard  display  size 
Pointing  Device 
No  video 
80386  processor 
No  simulation  capability 
Cue  augmentation 

The  capabilities  of  this  device  can  then  be  examined  in  light  of  its  ability  to  train 
the  above  skills.  If  the  requirements  of  skill  groups  are  not  adequately  met  by  the 
device  design,  then  the  designer  may  either  choose  a  more  sophisticated  device 
design,  or  decide  to  train  the  skill  groups  on  another  device. 


52 


After  completing  the  analysis  of  this  device,  the  process  would  be  employed 
for  other  device  categories  with  greater  cost.  The  recommended  device  design  for 
each  category  provides  the  input  to  the  allocate  training  resources  activity. 

Allocate  Training  Resources 

When  the  previous  activity  is  completed  for  all  skill-training 
strategies,  there  is  a  collection  of  skill-training  devices  defined  for  different 
strategies.  Although  skills  were  selected  because  of  the  potential  for  cost-effective 
skill  training,  we  would  not  expect  all  of  the  proposed  skill-training  devices  to 
provide  cost-effective  training  in  comparison  to  task-based  training  devices.  This 
activity  compares  the  projected  cost  of  training  using  task  training  only  to  the 
projected  cost  of  training  when  task  training  is  supplemented  by  skill-training. 

The  activity  proceeds  in  four  6teps.  The  first  two  steps  determine  the  cost 
of  task  training  and  the  cost  of  implementing  each  skill  training  strategy.  The 
analysis  of  skill  training  performed  in  the  second  6tep  also  determines  the  skill 
levels  produced  by  skill  training.  These  skill  levels  have  an  effect  on  the  entry 
performance  level  to  task  training,  thus  reducing  the  task-training  requirement. 
The  third  step  in  the  analysis  determines  the  impact  of  skill  training  on  the  cost 
of  subsequent  task  training.  Finally,  the  fourth  step  determines  those  skill¬ 
training  strategies  that  produce  a  net  improvement  in  overall  training  cost,  and 
selects  these  strategies. 
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A  MODEL  FOR  SKILL-BASED  TRAINING  DESIGN 


Introduction 


The  goal  of  the  model  is  to  combine  knowledge  about  the  nature  of  skills 
and  skill  acquisition  with  concepts  of  the  cost-effectiveness  of  skill  training  to 
produce  a  method  that  can  be  applied  to  problems  of  training  development  and 
training  system  design.  The  model  must  consider  (a)  the  goals  of  the  decision 
making  process  for  skill-based  training  system  design,  (b)  the  specific  training 
requirements,  (c)  the  nature  of  the  skills  required  for  the  job  being  trained,  and  (d) 
the  design  factors  that  decrease  cost  or  improve  effectiveness. 

As  a  first  6tep  in  the  model  development  process,  we  identified  who  the 
principal  users  are,  and  what  their  needs  are.  Given  this  knowledge,  we  then 
designed  the  model  with  the  needs  of  the  user  in  mind.  We  considered  the 
following  four  potential  users. 

1.  The  training  designer  responsible  for  determining  training  requirements 
and  producing  a  training  design, 

2.  The  training-device  designer,  responsible  for  converting  training 
requirements  into  training-device  design  specifications, 

3.  Research  laboratories,  responsible  for  developing  new  and  innovative 
solutions  to  training  problems,  and 

4.  Source  selection  evaluation  boards,  responsible  for  evaluating  the  training 
component  of  a  weapon  system  being  procured. 

Each  of  these  users  has  somewhat  different  concerns  regarding  skill-based 
training  options.  In  addition,  the  model  will  rely  on  a  subject-matter  expert,  such 
as  an  instructor  or  job  incumbent,  to  provide  some  of  the  required  input  data.  We 
considered  the  following  four  questions  to  represent  issues  that  could  be  addressed 
by  the  model. 

1.  Given  a  set  of  training  requirements,  which  skills  should  be  trained  isolated 
from  the  job  context? 

2.  Given  a  set  of  training  requirements  and  skills  to  be  trained,  what  training 
strategies  should  be  employed?  Training  strategies  specify  what  to  train, 
the  training  sequence,  and  the  criterion  performance  level.  They  form  the 
basis  of  training  device  requirements. 

3.  Given  a  skill-training  strategy,  what  interface  and  instructional  features 
should  be  incorporated  into  a  training  device  that  is  used  to  implement  the 
strategy? 
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4. 


Given  a  set  of  training  requirements,  skills,  and  training  devices,  how 
should  training  time  be  allocated  between  task  and  skill  training  on 
training  devices  or  actual  equipment  to  meet  the  training  requirements  for 
the  least  cost? 

Table  4  shows  our  assessments  of  which  questions  are  concerns  of  each  type  of 
user. 


Table  4 

Questions  of  Concern  to  Potential  Model  Users 


Questions 

Selection 

User  Category 

Training 

Designer 

Device 

Designer 

Research  Source 
Laboratory 

Select  Skills  for  Training 

X 

X 

X 

Select  Training  Strategies 

X 

X 

X 

Design  Training  Devices 

X 

X 

X  X 

Allocate  Training  Time 

X 

X 

In  developing  the  model,  we  took  the  viewpoint  of  the  training  designer, 
because  that  class  of  user  has  the  broadest  concern  with  skill-based  training 
system  design  issues.  The  design  engineer  also  had  many  of  these  concerns, 
depending  on  what  information  was  present  to  support  the  training  device 
requirement. 

The  process  of  model  development  involves  specification  of  model  processes 
and  data  in  increasing  levels  of  detail.  We  present  the  model  as  an  IDEFO  activity 
model.  The  IDEFO  model  describes  the  activities  included  in  the  model,  the  data 
required,  and  the  output  produced  by  the  model. 

The  section  continues  with  a  description  of  the  EDEFO  system  description 
methodology.  Then,  we  present  a  description  of  the  skill-based  design  model 
activities. 


56 


IDEFO  Methodology  Description 


IDEFO  (Integrated  Computer  Aided  Manufacturing  Definition)  was 
developed  by  the  Integrated  Computer  Aided  Manufacturing  Office  (ICAM)  of  the 
U.S.  Air  Force  to  be  used  as  a  tool  for  describing  the  functions  and  data  of  a 
complex  system  (SofTech,  Inc.,  1981;  Ross  &  Schoman,  1977).  A  system  consists  of 
any  combination  of  machinery  (hardware),  data,  and  people,  working  together  to 
perform  a  useful  function.  IDEFO  is  a  technique  that  enables  people  to 
understand  complex  systems  and  to  communicate  their  understanding  to  others. 
IDEFO  describes  the  functions  performed  by  the  system  by  successively 
decomposing  the  system  into  its  basic  components,  describing  how  each  component 
processes  information,  and  specifying  how  different  components  interact.  An 
IDEFO  model  is  expressed  as  a  series  of  related  diagrams;  each  diagram  describes 
a  particular  system  component  or  function.  An  IDEFO  diagram  is  composed  of 
boxes  and  arrows.  The  boxes  represent  component  functions  or  activities,  while 
the  arrows  represent  data  that  affect  the  activities  or  are  produced  by  them.  In 
this  report,  IDEFO  is  used  to  describe  the  components  and  functions  of  the  skill- 
based  training  system  design  model. 

IDEFO  Model  Organization 

The  diagrams  in  an  IDEFO  model  describe  the  system  in  a  modular, 
top-down  fashion,  showing  the  breakdown  of  the  system  into  its  component  parts. 
The  application  of  IDEFO  starts  with  the  most  general  or  abstract  description  of 
the  system  to  be  produced.  This  description  is  represented  in  a  diagram  as  a 
single  box;  that  box  is  subsequently  broken  down  into  a  number  of  more  detailed 
boxes,  each  of  which  represents  a  component  part.  The  component  parts  are  then 
detailed,  each  on  another  diagram.  Each  part  shown  on  a  detail  diagram  is  again 
broken  down,  and  so  forth,  until  the  system  is  described  to  the  desired  level  of 
detail.  Lower-level  diagrams,  then,  are  detailed  breakdowns  of  higher-level 
diagrams.  At  each  stage  of  breaking  down  the  system,  the  higher-level  diagram  is 
said  to  be  the  "parent"  or  overview  of  the  lower-level  "detail"  diagrams.  The 
relationship  between  diagrams  at  different  levels  is  shown  in  Figure  10. 

Diagram  display  format 

In  this  document,  each  diagram  in  an  IDEFO  model  is  displayed  in  a  two- 
page  format.  The  subject  diagram  is  shown  on  the  top  of  the  right-hand  page. 

The  parent  of  the  subject  diagram  is  shown  on  the  top  of  the  left-hand  page  with 
the  location  of  the  subject  node  indicated.  On  the  bottom  half  of  both  pages  is  text 
describing  the  operations  performed  by  each  activity  represented  in  the  diagram. 
Each  pair  of  pages  receives  a  page  number  that  is  displayed  as  part  of  the  subject 
diagram. 
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Diagram  node  numbers 

In  an  IDEFO  diagram,  the  component  parts  are  shown  as  numbered  boxes. 

A  diagram  should  have  no  more  than  six  boxes.  Each  box  at  one  level  is  detailed 
in  one  diagram  at  the  next  lower  level  until  a  sufficient  level  of  detail  is  reached. 
The  place  of  each  diagram  in  a  model  is  indicated  by  a  "node  number"  derived 
from  the  numbering  of  boxes.  For  example,  A21  is  the  diagram  that  details  box  1 
on  the  A2  diagram.  Similarly,  A2  details  box  2  on  the  AO  diagram,  which  is  the 
top  diagram  of  the  model.  The  parent  of  the  AO  diagram  represents  the  system  as 
a  single  box  and  is  denoted  "A-0."  The  hierarchy  may  be  shown  in  an  index  of 
diagram  names  and  their  node  numbers  called  a  "node  list."  The  node  list  serves 
as  a  table  of  contents  for  a  model.  In  an  IDEFO  model,  diagrams  are  displayed  in 
the  order  of  their  node  numbers. 

The  example  shown  in  Figure  11  provides  an  illustration  of  the  hierarchical 
decomposition  of  functions.  The  diagrams  in  Figure  11  indicate  that  the  overall 
function,  develop  system  (AO),  is  broken  down  into  three  sub-functions,  A1  through 
A3.  Design  system  (A2)  is  further  broken  down  into  three,  more  detailed 
sub-functions  (A21  through  A23). 

Description  of  Individual  IDEFO  Diagrams 

In  IDEFO,  boxes  represent  components  in  the  breakdown,  and  arrows 
represent  relationships  between  these  components.  Descriptive  labels  are  written 
inside  each  box  and  along  each  arrow  to  describe  their  meaning.  The  notation  is 
kept  simple  to  permit  easy  reading  with  little  special  training. 

Figure  12  shows  a  sample  IDEFO  diagram.  Notice  that  the  boxes  represent 
the  breakdown  of  activities  or  functions  performed  by  the  system  and  are  named 
by  verbs.  Arrows,  which  represent  objects  or  information,  are  labeled  with  nouns. 

Box-and-arrow  syntax 

The  sample  IDEFO  diagram  in  Figure  12  shows  that  the  descriptive  names 
and  labels  convey  the  box  and  arrow  contents  to  the  reader.  In  addition  to  its 
label,  the  side  at  which  an  arrow  enters  or  leaves  a  box  shows  its  role  as  an  input, 
control,  output,  or  mechanism  for  the  box  (see  Figure  13).  Arrows  that  enter  from 
the  left  of  an  activity  box  represent  inputs  to  the  process  represented  by  the  box. 
Inputs  represent  the  raw  materials  or  data  used  by  the  activity  to  produce 
outputs.  The  outputs  are  represented  by  arrows  that  originate  from  the  right  side 
of  the  box.  Arrows  entering  a  box  from  the  top  represent  controls  on  the  activity. 
Controls  are  data  that  provide  catalysts  or  constraints  for  the  represented  activity, 
but  are  not  changed  by  the  process.  Finally,  arrows  that  enter  a  box  from  the 
bottom  represent  mechanisms.  Mechanisms  are  the  agents  that  perform  the 
activities  represented  in  the  box.  In  short,  inputs  and  outputs  represent  what  is 
done  by  the  process,  controls  represent  why  it  is  done,  and  mechanisms  represent 
how  it  is  done. 
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LEVELS  OF  DIAGRAMS 
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Figure  11.  LD.LF0  node  numbering  convention. 
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Figure  12.  Sample  EDEFO  diagram. 


Figure  13.  Sample  IDEFO  diagram  showing  box  and  arrow  syntax. 
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The  arrow  structure  of  an  IDEFO  diagram  represents  a  constraint 
relationship  among  boxes.  It  does  not  represent  flow  of  control  or  sequence.  The 
arrows  entering  a  box  show  all  that  is  needed  by  the  box  to  perform  its  function; 
Therefore,  the  box  is  constrained  by  its  inputs  and  controls. 

Labeling  of  arrows 

Some  arrows  show  both  their  source  and  destination  boxes  on  the  same 
diagram,  while  others  have  one  end  unconnected  (see  Figure  14).  The 
unconnected  arrows  represent  inputs,  controls,  or  outputs  of  the  parent  box.  To 
find  the  source  or  destination  of  these  unconnected  arrows,  the  reader  must  locate 
the  matching  arrows  on  the  parent  diagram.  All  such  unconnected  arrows  must 
continue  on  the  parent  for  the  diagrams  to  be  complete. 

Although  arrow  connections  from  parent  boxes  to  detail  diagrams  are 
sometimes  obvious  from  the  labels,  the  IDEFO  methodology  includes  a  special 
notation  to  allow  readers  to  do  the  match  quickly.  In  this  notation,  unconnected 
arrows  are  numbered  according  to  the  position  of  those  arrows  in  the  parent 
diagram.  The  number  of  the  arrow  is  preceded  by  an  I,  C,  O,  or  M  to  indicate  that 
it  represents  an  input,  control,  output,  or  mechanism  in  the  parent  diagram. 

Thus,  if  an  arrow  is  labeled  "Cl,"  it  is  the  first  control  listed  in  the  parent 
diagram.  Similarly,  an  arrow  labeled  "03"  is  the  third  output  in  the  parent 
diagram. 

It  is  possible  for  a  data  element  to  serve  as  an  input  to  some  sub-activities 
of  a  given  activity  and  as  a  control  for  other  sub-activities.  In  this  case,  the  data 
are  represented  once  in  the  parent  diagram,  either  as  input  or  control.  In  the 
detailed  diagrams,  the  data  are  represented  as  a  control  in  some  diagrams  and  as 
an  input  in  others,  as  appropriate. 

Model  Formulation 

This  section  use6  IDEFO  to  describe  the  formulation  of  analysis  tools  to 
optimize  skill-based  training  systems.  This  problem  is  being  approached  from  the 
perspective  of  the  training  designer.  The  output  of  the  analysis  process  is  the 
specification  of  skill-based  training  strategies  and  devices  that  can  meet  the 
training  requirements  at  the  lowest  cost. 

A  significant  amount  of  data  will  be  required  to  operate  the  models 
proposed  here.  Some  of  these  data  will  be  gathered  from  the  research  literature 
and  reside  in  the  system’s  data  base.  Skill  and  task  analyses  will  also  produce 
some  of  the  required  data.  The  model  users,  who  are  the  training  developer  and 
subject  matter  expert,  will  be  required  to  provide  the  rest  of  the  data. 

In  order  to  facilitate  the  description  the  optimization  tools  fo  •  skill-based 
training  systems,  three  tables  are  included  with  the  IDEFO  diagrams: 
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1.  A  list  of  the  nodes  in  the  structure  in  the  order  that  they  appear  in  the 
system  description.  The  node  list  provides  the  table  of  contents  for  the 
IDEFO  model.  If  the  node  is  represented  by  its  own  diagram,  the  number  of 
that  diagram  is  listed  in  the  final  column  of  the  node  list.  Nodes  that  have 
no  detailed  diagram  do  not  have  a  node  number  listed.  The  descriptions  of 
such  a  node  may  be  found  on  the  diagram  for  its  parent  node. 

2.  A  list  of  the  model  outputs  indicating  the  source  node  and  all  destination 
nodes  for  each  output.  A  destination  listed  as  "UP"  indicates  that  the 
output  is  unconnected  in  a  particular  IDEF  diagram.  The  output  map 
specifies  whether  each  output  is  used  as  an  input  or  control  at  the 
destination  node.  Each  output  description  is  referenced  to  a  single  IDEF 
diagram  number  as  indicated  in  the  output  map. 

3.  A  data  base  map  that  illustrates  all  data  requirements  and  mechanisms 
used  by  the  model.  The  data  base  map  lists  all  node  numbers  in  which  each 
data  element  or  mechanism  is  used,  and  specifies  whether  the  data  are  used 
as  an  input,  control,  or  mechanism  for  each  node. 

A  description  of  a  single  node  in  the  model  consists  of  three  components: 
two  diagrams  (which  may  be  repeated)  and  associated  explanatory  text.  The 
diagram  for  the  node  being  described  is  on  the  right-hand  side  of  the  page;  its 
parent  is  shown  on  the  left-hand  side.  The  text  is  written  beneath  the  diagrams. 

If  the  explanatory  description  requires  more  than  two  pages,  both  parent  and  child 
diagrams  are  repeated  on  the  next  two  pages,  until  the  text  is  completed. 
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EDEF  Node  List 


Node 


Title 


A-0 

Specify  Skill-Based  Training  Strategies 
and  Devices 

AO 

Specify  Skill-Based  Training  Strategies 
and  Devices 

A1 

Identify  Skills  for  Training 

All 

Select  Tasks  for  Analysis 

A12 

Identify  Possible  General  Abilities 

A121 

Determine  Level  of  Analysis 

A122 

Review  Task-Element  Verbs 

A123 

List  Possible  Abilities 

A13 

Select  Required  Abilities 

A131 

Determine  Distinguishing  Questions 

A132 

Select  Abilities  Required  for  Performance 

A133 

Verify  Performance  Gap 

A14 

Identify  Specific  Skills 

A141 

Select  Skill  Frames 

A142 

Incorporate  Task-Element  Objects 

A143 

Incorporate  Domain  Specific  Knowledge 

A144 

Select  Completed  Skill  Definitions 

A15 

Determine  Skill  Importance 

A151 

Calculate  Raw  Score 

A152 

Determine  Sum  Over  Skills 

A153 

Divide  Raw  Score  by  Sum 

A2 

Select  Skill-Training  Strategies 

A21 

Group  Skills 

A211 

Calculate  Object-Set  Least  Upper  Bound 

A212 

Partition  Skills  to  Groups 

A213 

Calculate  Group  Weight 

A214 

Select  Top  Skill  Groups 

A22 

Incorporate  Training  Criterion 

A221 

Determine  Maximum  Criterion 

A222 

Assess  Skill  Training  Factors 

A223 

Set  Skill  Training  Criterion 

A23 

Incorporate  Training  Methods 

A231 

Select  Part  or  Whole  Methods 

A232 

Recommend  Specific  Method 

A24 

Determine  Interface  Requirements 

A241 

Select  Relevant  Interface  Dimensions 

A242 

Perform  Preliminary  Restrictions 

EDEF 

Number 

SB001 

SB002 

SB003 

SB004 

SB005 

SB006 

SB007 

SB008 

SB009 

SB010 

SB011 

SB012 
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IDEF  Node  List  (Cont.) 


Node 


Title 


IDEF 

Number 


A243 

A25 

A3 

A31 

A32 

A321 

A322 

A323 

A324 

A325 

A33 

A331 

A332 

A3321 

A3322 

A3323 

A3324 

A333 

A3331 

A3332 

A3333 

A34 


Determine  Required  Levels 
Determine  Instructor  Support  Requirements 
Design  Skill-Training  Devices  SB013 

Organize  Skill  Groups  by  Device  Requirements 
Select  Instructional  Features  by  SB014 

Device  Category 
Calculate  Effectiveness  Score 
Calculate  Feature  Benefit 
Compute  Benefit/Cost  Ratios  for  Options 
Sort  Options  by  Benefit/Cost  Ratios 
List  Optimal  Features 

Select  Interface  Features  by  Device  Category  SB015 
Construct  Training  Device  Options 
Calculate  Cost  and  Benefit  of  Options  SB016 

Calculate  Costs 
Determine  Skill  Trainability 
Calculate  Benefit  Scores 
Calculate  Benefit  Weights 

Compute  Optimal  Device  Designs  SB017 

Compute  Benefit/Cost  Ratios  for  Options 
Sort  Options  by  Benefit/Cost  Ratios 
Compute  Device  Designs 
Recommend  Device  Designs 


A4 

A41 

A411 

A412 

A42 

A421 

A422 

A423 

A43 

A431 

A432 

A433 

A44 

A441 

A442 

A443 


Allocate  Training  SB018 

Detail  Skill  Interaction  with  Task  Training  SB019 

Characterize  Baseline  Training  and  Costs 
Characterize  Skill  Impact  on  Training 
Determine  Per-Student  STS/Device  Costs  SB020 

Determine  Skill  Learning  Curves  for  Device 
Determine  Required  STS  Training  Time  by  Device 
Determine  Per-Student  Cost  by  STS/Device 
Determine  Impact  on  Task  Training  Costs  SB021 

Determine  Training  Cost  without  STS/Device 
Determine  Training  Cost  with  STS/Device 
Determine  Training  Cost  Savings  with  STS/Device 
Choose  Next  STS/Device  for  Greatest  Payoff  SB022 
Construct  Payoff  Matrix 
Determine  Payoff  for  Each  STS/Device 
Select  STS/Device  with  Greatest  Payoff 
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IDEF  Output  Map 


Description 

Source 

Destin¬ 

ation 

Function 

IDEF 

Number 

Specific  Skills 

A1 

UP 

SB002 

Specific  Skills 

A1 

A 2 

Input 

SB002 

Specific  Skills: 

Specific  Skills 

A14 

UP 

SB003 

Specific  Skills 

A14 

A15 

Input 

SB003 

Specific  Skills 

A144 

UP 

SB006 

Skill  Importance  Weight 

A15 

UP 

SB003 

Skill  Importance  Weight 

A153 

UP 

SB007 

Selected  Tasks: 

Task  Element  Verbs 

All 

A12 

Input 

SB003 

Task  Element  Objects 

All 

A13 

Control 

SB003 

Task  Element  Objects 

All 

A14 

Control 

SB003 

Training  Time  &  Criticality 

All 

A16 

Control 

SB003 

Possible  General  Abilities 

A12 

A13 

Input 

SB003 

Possible  General  Abilities 

A123 

UP 

SB004 

Genera]  Abilities  Required 

For  Training 

A13 

A14 

Input 

SB003 

General  Abilities  Required 

For  Training 

A133 

UP 

SB005 

Selected  Task-Element  Verbs 

A121 

A122 

Input 

SB004 

Revised  Task-Element  Verbs 

A122 

A123 

Control 

SB004 

Distinguishing  Features 

A131 

A132 

Control 

SB005 

Abilities  Required  for 

Performance 

A132 

A133 

Input 

SB005 

Selected  Skill  Frames 

A141 

A142 

Input 

SB006 

Partial  Skill  Definitions 

A142 

A143 

Input 

SB006 

Elaborated  Skill  Definitions 

A143 

A 144 

Input 

SB  006 

Raw  Skill  Importance 

A151 

A152 

Input 

SB007 

Raw  Skill  Importance 

A151 

A153 

Input 

SB007 

Normalization  Factor 

A152 

A153 

Input 

SB007 

Device  Requirements 

A2 

A3 

Control 

SB002 

Device  Requirements: 

Interface  Requirements 

A24 

UP 

SB008 

Interface  Requirements 

A243 

UP 

SB012 

Instructor  Support 

Requirements 

A25 

UP 

SB008 

Training  Strategies 

A2 

UP 

SB002 

Training  Strategies 

A2 

A3 

Input 

SB002 

Training  Strategies 

A2 

A4 

Input 

SB002 

Training  Strategies 

A23 

UP 

SB008 
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Destin-  IDEF 


Descrintion  Source 

ation 

Function 

Number 

Training  Strategies 

A23 

A24 

Control 

SB008 

Training  Strategies 

A23 

A25 

Control 

SB008 

Training  Strategies 

A232 

UP 

SB011 

Selected  Skill  Groups 

A21 

A22 

Input 

SB008 

Selected  Skill  Groups 

A214 

UP 

SB009 

Partial  Training  Strategies 

A22 

A23 

Input 

SB008 

Partial  Training  Strategies 

A223 

UP 

SB010 

Least  Upper  Bounds 

A211 

A212 

Control 

SB009 

Skill  Groups 

A212 

A213 

Input 

SB009 

Skill  Groups 

A212 

A214 

Input 

SB009 

Skill  Group  Weights 

A213 

A214 

Control 

SB009 

Maximum  Criterion 

A221 

A223 

Control 

SB010 

Skill  Training  Advantage 

A222 

A223 

Control 

SB010 

Whole-Part  Recommendations 

A231 

A232 

Input 

SB011 

Relevant  Dimensions 

A241 

A242 

Input 

SB012 

Relevant  Dimensions 

A241 

A243 

Input 

SB012 

Level  Restrictions 

A242 

A243 

Input 

SB012 

Training  Device  Designs 

A3 

UP 

SB002 

Training  Device  Designs 

A3 

A4 

Input 

SB002 

Training  Device  Designs 

A34 

UP 

SB013 

SkiiLDevice  Matrix 

A31 

A3  2 

Input 

SB013 

Skill/Device  Matrix 

A31 

A33 

Input 

SB013 

Skill/Device  Matrix 

A31 

A34 

Input 

SB013 

Selected  Instructional  Features 

A32 

A34 

Input 

SB013 

Selected  Instructional  Features 

A325 

UP 

SB  014 

Selected  Interface  Features 

A3  3 

A34 

Input 

SB013 

Selected  Interface  Features 

A333 

UP 

SB015 

Selected  Interface  Features 

A3333 

UP 

SB017 

Weighted  Feature 

Effectiveness 

A321 

A322 

Input 

SB014 

Feature  Benefit 

A322 

A3  23 

Input 

SB014 

Benefit/Cost  Ratio 

A323 

A324 

Control 

SB014 

Benefit/Cost  Priority 

A3 24 

A325 

Control 

SB014 

Training  Device  Options 

A331 

A332 

Input 

SB015 

Training  Device  Options 

A331 

A333 

Input 

SB015 

Cost  and  Benefit  of  Options 

A332 

A333 

Control 

SB015 

Cost  and  Benefit  of  Options: 

Option  Costs 

A3321 

UP 

SB016 

Option  Benefit  Scores 

A3323 

UP 

SB016 

Option  Benefit  Scores 

A3323 

A3324 

Input 

SB016 

IDEF  Output  Map  (Cont.) 


Description 

Source 

Destin¬ 

ation 

Function 

IDEF 

Number 

Fidelity  Dimension  Weights 

A3324 

UP 

SB016 

Skills  Trained  by  Option 

A3322 

A3323 

Input 

SB016 

Benefit/Cost  Ratio  for  Options 
Benefit/Cost  Priority  of 

A3331 

A3332 

Control 

SB017 

Options 

A3332 

A3333 

Control 

SB017 

Training  Allocation 

A4 

UP 

SB002 

Training  Allocation 

Training  Allocation: 

A4 

A3 

Control 

SB002 

Selected  Strategies/Devices 

A44 

UP 

SB018 

Selected  Strategy/Device 

A443 

UP 

SB022 

Skill  Impact  Parameter  Values 

A41 

A43 

Control 

SB018 

Skill  Impact  Parameter  Values 

A412 

UP 

SB019 

Baseline  Training  Cost 

A41 

A43 

Control 

SB018 

Baseline  Training  Cost 

A411 

UP 

SB019 

Skill  Levels  Achieved 

A42 

A43 

Input 

SB018 

Skill  Levels  Achieved 

A42 

A44 

Input 

SB018 

Skill  Levels  Achieved 

A424 

UP 

SB020 

Strategy/Device  Cost 

A42 

A44 

Input 

SB018 

Strategy/Device  Cost 

A423 

UP 

SB020 

Per-Student  Cost  Savings 

A43 

A44 

Control 

SB018 

Per-Student  Cost  Savings 

A433 

UP 

SB021 

Pre-Task  Training  Skill  Levels 

A44 

A42 

Control 

SB018 

Pre-Task  Training  Skill  Levels 

A44 

A43 

Control 

SB018 

Pre-Task  Training  Skill  Levels 

A443 

UP 

SB022 

Revised  Baseline  Training  Cost 

A44 

A43 

Control 

SB018 

Revised  Baseline  Training  Cost 
Baseline  Task  Training 

A443 

UP 

SB022 

Description 

Skill  Learning  Curves  by 

A411 

A412 

Input 

SB019 

Strategy/Device 

Training  Times  by  Strategy/ 

A421 

A422 

Input 

SB020 

Device 

A422 

A423 

Input 

SB020 

Cost  without  Strategy/Device 

A431 

A433 

Input 

SB021 

Cost  with  Strategy/Device 

A432 

A433 

Input 

SB021 

Skill  Training  Payoff  Matrix 

A441 

A442 

Input 

SB022 

STS/Device  Payoffs 

A442 

A443 

Input 

SB022 
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Data  Base  Map 


IDEF 

Description _ Destination _ Function  Number 


Task  Data 

AO 

Input 

SB001 

A1 

Input 

SB002 

A2 

Input 

SB002 

A4 

Input 

SB002 

All 

Input 

SB003 

Task  Training  Criteria 

A22 

Control 

SB008 

A221 

Control 

SB010 

Task  Training  Device  Fidelities  and 
Instructional  Features 

A41 

Input 

SB018 

A411 

Input 

SB019 

Task  Training  Sequence 

A41 

Input 

SB018 

A411 

Input 

SB019 

Skill  List 

AO 

Input 

SB001 

(Skill  Data) 

A1 

Input 

SB002 

General  Ability  List 

A12 

Input 

SB003 

A123 

Input 

SB004 

Device  Design  Options 

AO 

Input 

SB001 

A2 

Input 

SB002 

A3 

Input 

SB002 

Interface  Options 

A24 

Input 

SB008 

A241 

Input 

SB012 

A33 

Input 

SB013 

A331 

Input 

SB015 

Instructor  Support  Options 

A25 

Input 

SB008 

A32 

Input 

SB013 

A321 

Input 

SB014 

A324 

Input 

SB014 

A325 

Input 

SB014 

Benefit  Weight 

A322 

Input 

SB014 

Instructional  Feature  Cost 

A323 

Input 

SB014 
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Data  Base  Map  (cont.) 


EDEF 


Description 

Destination 

Function 

Number 

Task  Training  Device  Cost  Data 

A411 

Input 

SB019 

A412 

Input 

SB019 

Semantic  Information 

AO 

Control 

SB001 

A1 

Control 

SB002 

Verb  Hierarchy 

A12 

Control 

SB003 

A121 

Control 

SB004 

A122 

Control 

SB004 

Verb- Ability  Associations 

A12 

Control 

SB003 

A122 

Control 

SB004 

A123 

Control 

SB004 

Learner  Characteristics 

AO 

Control 

SB001 

A1 

Control 

SB002 

A2 

Control 

SB002 

A4 

Control 

SB002 

All 

Control 

SB003 

A41 

Control 

SB018 

A42 

Control 

SB018 

A412 

Control 

SB019 

Entry  Skill  Level 

A13 

Control 

SB003 

A133 

Control 

SB005 

A23 

Control 

SB008 

A231 

Control 

SB011 

A421 

Control 

SB020 

Number  of  Students  Annually 

A423 

Control 

SB020 

Domain  Information 

AO 

Control 

SB001 

A1 

Control 

SB002 

A2 

Control 

SB002 

A3 

Control 

SB002 

A13 

Control 

SB003 

A14 

Control 

SB003 

A132 

Control 

SB005 

A143 

Control 

SB006 

A21 

Control 

SB008 

A24 

Control 

SB008 
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ft 


Data  Base  Map  (coni.) 


IDEF 


Description 

Destination 

Function 

Number 

Domain  Information  (cont.) 

A25 

Control 

SB  008 

A242 

Control 

SB012 

A243 

Control 

SB012 

Object  Hierarchy 

A211 

Control 

SB009 

Training  Device  Information 

A31 

Control 

SB013 

A32 

Control 

SB013 

A33 

Control 

SB013 

A321 

Control 

SB014 

A331 

Control 

SB015 

Resource  Constraints 

AO 

Control 

SB001 

A3 

Control 

SB002 

A34 

Control 

SB013 

Completion  Criteria 

A144 

Control 

SB006 

User-Selected  Number  of  Groups 

A214 

Control 

SB009 

Skill  Impact  Evaluation  Framework 

A41 

Control 

SB018 

A43 

Control 

SB018 

A412 

Control 

SB019 

A432 

Control 

SB021 

Training  Designer 

AO 

Mechanism 

SB001 

Al 

Mechanism 

SB002 

A2 

Mechanism 

SB002 

A3 

Mechanism 

SB002 

All 

Mechanism 

SB003 

A12 

Mechanism 

SB003 

A121 

Mechanism 

SB004 

A122 

Mechanism 

SB004 

A21 

Mechanism 

SB008 

A214 

Mechanism 

SB009 

A32 

Mechanism 

SB  013 

A33 

Mechanism 

SB013 

A34 

Mechanism 

SB013 

A325 

Mechanism 

SB014 

A331 

Mechanism 

SB015 

A333 

Mechanism 

SB015 
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Data  Base  Map  (cont.) 


Description 

Destination 

Function 

IDEF 

Number 

Training  Designer  (cont.) 

A3333 

Mechanism 

SB017 

SME 

AO 

Mechanism 

SB001 

A1 

Mechanism 

SB002 

A2 

Mechanism 

SB002 

A4 

Mechanism 

SB002 

A12 

Mechanism 

SB003 

A13 

Mechanism 

SB003 

A14 

Mechanism 

SB003 

A122 

Mechanism 

SB004 

A132 

Mechanism 

SB005 

A133 

Mechanism 

SB005 

A143 

Mechanism 

SB006 

A22 

Mechanism 

SB008 

A23 

Mechanism 

SB008 

A24 

Mechanism 

SB008 

A25 

Mechanism 

SB008 

A221 

Mechanism 

SB010 

A222 

Mechanism 

SB010 

A231 

Mechanism 

SB011 

A232 

Mechanism 

SB011 

A  9.43 

Mechanism 

SB012 

A41 

Mechanism 

SB018 

A411 

Mechanism 

SB019 

A421 

Mechanism 

SB020 

Optimization  Methods 

AO 

Mechanism 

SB001 

OSBATS  Methodology 

A4 

Mechanism 

SB002 

A41 

Mechanism 

SB018 

A42 

Mechanism 

SB018 

OSBATS  Learning  Curve  Con¬ 

struction  Methodology 

A411 

Mechanism 

SB019 

A421 

Mechanism 

SB020 

OSBATS  Training  Device 

Selection  Methodology 

A412 

Mechanism 

SB019 

OSBATS  Device  Costing 

Methodology 

A423 

Mechanism 

SB020 
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System  Description 

SB/A-0:  Specify  Skill-Based  Training  Strategies  and  Devices 

The  purpose  of  the  Skill-Based  Optimization  of  Training  model  is  to  specify 
the  skill -training  strategies  and  devices  that  meet  training  requirements  at  the 
lowest  cost.  Training  effectiveness  is  measured  by  the  level  of  performance  on 
designated  tasks.  Cost  has  investment,  fixed  operating  and  variable  operating 
components. 

This  analysis  breaks  the  specification  of  skill-based  training  strategies  and 
devices  into  four  steps  and  describes  the  processes  that  comprise  these  steps.  The 
1DEF0  diagrams  describe  the  solution  process  from  the  perspective  of  the  training 
designer. 

The  IDEFO  analysis  assumes  that  training  requirements  are  defined  by  a 
set  of  tasks  that  must  be  performed  to  prescribed  standards  at  the  conclusion  of 
the  training  period.  First,  the  specific  skills  required  to  meet  task  performance 
criteria  are  identified.  Next,  skill-training  strategies  are  developed  for  each  skill. 
Then  skill  training  devices  are  designed  to  implement  each  strategy.  Finally  a 
cost-benefit  analysis  is  conducted  to  identify  which  skill  training  strategies  and 
devices  lower  the  cost  of  training  from  that  established  by  a  solution  (i.e., 

OSBATS)  that  is  purely  oriented  to  task  training. 

Input  Data.  Input  data  consist  of  the  task  data,  skill  data,  and  training 
device  options  (including  instructional  features  and  interface  elements).  Task  data 
include  information  typically  produced  by  a  task  analysis,  including  descriptions  of 
tasks,  subtasks,  and  task  elements,  as  well  as  ratings  of  task  criticality,  frequency 
of  performance,  and  required  training  time.  Skill  data  include  a  list  of  general 
abilities,  skill  definitions  used  to  distinguish  different  abilities,  and  frameworks 
for  building  specific  skills  from  general  abilities.  Device  design  options  include 
cost  and  performance  information  about  the  interface  and  instructional 
components  that  could  form  the  components  of  a  skill-based  training  device. 

Controls.  Three  types  of  data  control  the  solution  process:  resource 
constraints,  learner  characteristics,  and  domain  information.  Resource  constraints 
specify  the  amount  of  money  available  for  training-device  development  or  the  per- 
student  life  cycle  cost  constraint  for  training.  Learner  characteristics  establish  the 
entry-level  skills  and  task  performance  levels  of  the  students.  Domain 
information  describes  characteristics  of  the  job  being  trained,  such  as  equipment 
used,  displays  and  controls,  and  specific  features  of  job  performance. 

Mechanisms.  Mechanisms  employed  in  this  solution  process  are  the  users 
of  the  model  and  optimization  methods.  We  distinguish  two  user  roles,  training 
designer  and  subject-matter  expert.  Optimization  methods  are  employed  by  all 
subactivities;  consequently,  this  mechanism  will  not  be  showm  in  the  descriptions 
of  lower-level  activities. 
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Specify  Skill-Based  Training  Strategies  and  Devices 


Output.  The  primary  output  of  this  process  is  an  allocation  of  training  time 
to  skills  and  tasks  on  the  relevant  devices  for  each.  This  allocation  of  training 
achieves  a  lower  cost  than  would  be  obtained  by  training  tasks  alone.  Other 
outputs  include  the  skills  that  need  to  be  trained,  the  training  strategy  for  each 
skill,  and  a  definition  of  the  training  device  that  is  used  to  implement  each 
skill-based  training  strategy. 
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SB/AO:  Specify  Skill-Based  Training  Strategies  and  Devices 


This  overall  model  process  consists  of  four  major  activities:  (a)  identifying 
the  specific  skills  required  for  job  performance,  (b)  organizing  skills  to  determine 
training  strategies,  (c)  designing  training  devices  to  implement  the  skill-training 
strategies,  and  (d)  allocating  training  time  to  skill  and  task  training  to  minimize 
the  training  cost  needed  to  achieve  the  required  task  performance. 


SB/A1:  Identify  skills  for  training.  This  activity  identifies  the  specific  skills 
needed  to  achieve  training  requirements.  First,  the  tasks  that  offer  the  greatest 
potential  for  skill  training  are  identified.  Then  general  abilities  are  matched  to 
task  activities,  and  the  required  abilities  are  identified.  The  general  abilities  are 
defined  in  greater  detail  to  produce  specific  skills.  Finally,  the  relative  importance 
of  the  specific  "kills  for  training  each  task  are  determined. 


SB/A 2:  Select  skill-training  strategies.  This  activity  defines  skill -training 
strategies  (STSs),  which  consist  of  a  group  of  skills  to  be  trained,  a  training 
criterion,  and  a  sequencing  method.  In  addition,  the  activity  defines  interface  and 
instructor  support  requirements  for  each  skill-training  strategy.  Skills  are 
grouped  according  to  the  to  objects  on  which  they  operate  and  according  to  the 
general  ability  category  required.  The  skill  training  criterion  incorporates 
information  about  relevant  task  training  criteria,  as  well  as  characteristics  of  the 
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skill.  The  sequencing  options  are  recommended  based  upon  the  entry  skill  level, 
and  particular  sources  of  difficulty  in  skill  performance.  The  training  strategy 
controls  the  selection  of  interface  and  instructor  support  requirements. 


SB/A3:  Design  Devices.  This  activity  applies  a  cost-Hj’iL*  analysis  to 
specify  the  interface  levels  and  instructional  features  that  provide  the  maximum 
enhancement  per  development  dollar.  The  activity  is  applied  individually  for  each 
class  of  skill-training  device  being  designed.  In  this  activity,  6kill-training 
strategies  are  initially  organized  according  to  their  device  requirements.  Optimal 
instructional  features  and  interface  levels  are  then  determined  using  a  cost-benefit 
analysis.  The  recommendations  regarding  instructional  features  and  interface 
levels  are  then  combined  to  recommend  a  skill-training  device  design. 

SB/A4:  Allocate  Training.  This  activity  uses  cost-benefit  optimization  to 
find  the  lowest  training  cost  using  a  combination  of  skill  and  task  training.  The 
skill-training  strategies  and  their  devices  are  first  ordered  according  to  their 
cost-effectiveness.  Then  the  list  is  searched  to  find  the  break  even  point  where 
additional  skill  training  is  no  longer  cost-effective  compared  to  task  training. 
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SB/A1:  Identify  Skills  for  Training 


This  activity  produces  a  list  of  specific  skills  required  to  achieve  training 
requirements  defined  by  a  set  of  tasks  with  standards,  given  the  entry  level 
performance  capabilities  of  the  student  population,  information  about  the  job 
domain,  and  knowledge  of  the  procedures  that  were  used  to  describe  tasks  and 
their  components.  After  those  tasks  for  which  skill  training  is  not  applicable  are 
eliminated,  general  abilities  are  associated  with  each  remaining  task.  Each 
general  ability  identified  is  then  defined  in  terms  of  the  specific  actions  and 
objects  of  these  actions  by  which  the  ability  is  instantiated.  Finally  the  relative 
importance  of  the  specific  skills  for  training  each  task  is  determined. 


SB/All:  Select  tasks  for  analysis.  This  activity  examines  the  tasks 
composing  the  training  requirement  to  select  those  tasks  for  which  skill-based 
training  is  appropriate.  Factors  examined  include:  (a)  entry  level  performance — 
skill  training  is  most  effective  if  the  students  entry  performance  level  is  low;  (b) 
total  task  training  time  on  the  actual  equipment — if  a  task  requires  a  great  deal  of 
training,  there  is  a  large  potential  for  benefits  from  skill  training;  and  (c) 
workload— if  the  task  workload  is  great  (e.g.,  the  task  requires  time  sharing 
among  multiple  activities),  then  skill  training  may  reduce  the  workload  during 
training.  The  data  required  for  this  activity  are  assumed  to  be  available  from 
existing  task  analyses.  This  activity  will  assess  the  values  of  relevant  data 
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variables,  calculate  a  weighted  sum  of  the  multiple  variables  to  produce  a  single 
index,  and  select  those  tasks  for  which  the  index  is  greater  than  a  designer- 
specified  criterion  value.  The  output  is  a  list  of  tasks  that  are  suited  for  skill 
training. 


SB/A12:  Identify  possible  general  abilities.  This  activity  identifies  a  set  of 
general  abilities  that  includes  the  abilities  actually  involved  in  performing  the 
tasks.  The  abilities  are  chosen  from  a  list  of  general  abilities  based  on  the 
description  of  the  action  involved  in  each  task  element.  The  selection  of  abilities 
is  based  on  associations  that  tie  each  verb  in  a  task  element  description  to  one  or 
more  general  abilities.  In  general,  this  activity  will  generate  more  general 
abilities  than  are  actually  required  by  the  tasks.  Irrelevant  abilities  are 
eliminated  in  the  next  two  processes. 

SB/A13:  Select  required  abilities.  This  activity  selects  the  general  abilities 
that  are  required  for  training  from  the  candidate  abilities  identified  in  the 
previous  process.  The  selection  is  based  on  the  objects  the  task-element  actions, 
and  knowledge  about  the  job  domain.  Some  of  the  required  job  knowledge  will  be 
provided  by  a  subject-matter  expert  (SME),  such  as  an  instructor  or  job 
incumbent.  The  abilities  required  for  training  will  be  those  selected  general 
abilities  with  sufficiently  low  entry  skill  level. 
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SB/A14:  Identify  specific  skills.  This  activity  adds  detail  to  the  general 
abilities  that  were  identified  in  the  previous  activity  to  produce  a  description  of 
the  specific  skills  required  for  training.  The  type  of  general  ability  (e.g.,  pattern 
recognition)  determines  what  kind  of  information  is  required  to  produce  a  specific 
skill  description  (e.g.,  the  sensory  modality,  the  pattern  that  must  be  recognized, 
the  background  in  which  the  pattern  is  located,  a  description  of  conditions,  etc.). 
The  required  information  may  come  from  the  objects  of  the  task-element  actions, 
or  domain  information.  Some  of  the  required  job  knowledge  will  be  provided  by 
SMEs.  The  output  of  this  activity  is  a  set  of  definitions  of  the  specific  skills 
required  for  job  performance. 


SB/A15:  Determine  skill  importance.  The  previous  activities  have  defined 
and  refined  a  matrix  that  relates  skills  to  those  tasks  that  can  be  reasonably 
trained  to  higher  performance  levels  through  skill  training.  This  activity  assigns 
relative  importance  across  the  skills  for  enhancing  the  performance  of  a  particular 
task.  The  importance  weight  will  be  based  on  the  results  of  a  task  analysis:  task 
training  time,  task  criticality,  and  task-element  criticality.  The  output  of  this 
activity  is  a  set  of  weights  that  express  the  relative  importance  of  each  skill 
identified  in  the  analysis. 
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SB/A12:  Identify  Possible  General  Abilities 


This  activity  lists  general  abilities  that  may  be  involved  in  a  job,  based  on  a 
description  of  task-element  actions  that  is  part  of  a  task  analysis.  The  first  two 
steps  in  this  activity  review  the  task  analysis  to  ensure  a  consistent  and 
appropriate  level  of  detail  in  the  verbs  used  to  describe  task-element  actions. 

SB/A121:  Determine  level  of  analysis.  Task  analyses  divide  tasks  into 
subtasks  and  task  elements.  This  activity  examines  the  subtask  and  task-element 
descriptions  and  compares  the  verbs  used  in  these  descriptions  to  an  internal  verb 
hierarchy.  The  verbs  used  to  describe  subtasks  and  task  elements  will  have  an 
influence  on  the  output  of  this  activity.  The  verbs  should  be  as  specific  as 
possible;  that  is,  they  should  be  associated  with  as  few  skills  as  possible.  When  a 
subtask  verb  is  at  the  lowest  level  of  the  verb  hierarchy,  then  the  system 
recommends  analysis  at  the  subtask  level.  The  system  also  recommends  analysis 
at  the  subtask  level  when  the  subtask  is  procedural.  When  the  subtask  verb  is  at 
an  intermediate  level  of  the  verb  hierarchy,  and  the  related  task-element  verbs 
are  at  lower  level,  then  the  system  will  recommend  analysis  at  the  task-element 
level  for  the  elements  of  that  subtask.  The  system  will  present  the 
recommendations  to  the  training  designer,  who  will  either  accept  or  overrule 
them. 
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SB/A122:  Review  task-element  verbs.  This  step  reviews  the  subtask  or 
task-element  statements  that  are  generated  in  the  previous  step.  If  the  verbs  are 
not  at  the  lowest  level  of  the  verb  hierarchy  and  the  verbs  are  associated  with 
more  than  one  general  ability  on  the  internal  verb-by-ability  association  matrix, 
then  the  SME  is  prompted  to  provide  a  more  specific  description  of  the  task 
element.  For  example,  the  verb  "synthesize"  may  require  any  of  several  skills.  If 
this  verb  were  used  to  characterize  a  task  element,  the  SME  would  be  prompted  to 
give  a  more  precise  description  of  the  action,  such  as  "integrate."  This  step  is 
conducted  to  minimize  the  number  of  possible  general  abilities  that  are  generated 
in  the  next  step  of  the  analysis. 

SB/A123:  List  possible  general  abilities.  This  step  lists  the  possible  general 
abilities,  based  on  a  verb-by-ability  matrix  that  associates  each  available  task- 
element  verb  with  one  or  more  general  abilities. 
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This  activity  selects  the  general  abilities  that  are  required  for  training, 
based  on  the  candidate  abilities  that  were  generated  by  the  previous  activity 
(A12).  The  activity  uses  information  contained  in  task-element  descriptions, 
information  about  the  job  domain,  and  other  information  provided  by  the  SME  to 
select  those  skills  that  are  actually  required  for  job  performance.  The  abilities 
required  for  performance  for  which  students  have  a  low  entry  skill  level  are 
required  for  training. 

SB/A131:  Determine  distinguishing  questions.  This  activity  determines  the 
characteristics  that  can  be  used  to  distinguish  the  alternative  general  abilities 
that  may  be  involved  in  a  task  element.  The  characteristics  are  based  on  the 
general  ability  definitions.  Each  general  ability  is  defined  by  a  set  of  rules  with 
conditions  that  specify  the  kinds  of  activities,  objects,  and  domain  characteristics 
that  are  associated  with  each  general  ability.  This  activity  examines  the 
conditions  for  all  general  ability  candidates  produced  by  A123  and  identifies  those 
conditions  that  distinguish  the  candidate  abilities. 

SB/A132:  Select  abilities  required  for  performance.  This  activity  examines 
available  information  about  the  task-element  objects,  and  domain  characteristics 
as  controlled  by  the  conditions  identified  in  A131  to  select  the  general  abilities 
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that  are  required  for  performance  from  the  candidates  identified  in  A 123.  Where 
the  existing  task-analytic  information  is  not  sufficient  to  distinguish  alternative 
skills,  the  SME  will  be  asked  to  provide  additional  information.  The  output  of  this 
activity  is  a  set  of  abilities  that  are  required  for  job  performance. 


SB/A133:  Verify  performance  gap.  In  this  activity,  the  SME  will  be  asked 
to  provide  the  entry  skill  level  for  each  general  ability.  The  task  entry 
performance  level  will  be  used  as  an  anchor  to  obtain  the  skill  level,  which  will  be 
measured  on  the  same  scale  as  entry  task  performance.  Those  abilities  for  which 
the  student  is  already  competent  are  eliminated  from  further  analysis.  The 
remaining  abilities  are  those  for  which  training  is  required. 
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SB/A14:  Identify  Specific  Skills 


This  activity  converts  the  general  abilities  required  for  training  into  more 
detailed  descriptions  of  specific  skill  requirements.  Each  general  ability  is 
associated  with  a  frame  or  template  that  specifies  the  kind  of  information  that  is 
required  to  develop  a  complete  definition  of  a  specific  instance  of  that  general 
ability.  Information  to  complete  the  definition  comes  from  the  task  analysis,  job 
domain  information,  and  SME  knowledge.  It  is  possible  that  general  skills  that 
were  incorrectly  identified  in  the  previous  activity  (A13)  were  not  eliminated  by 
the  checks  performed  in  that  activity.  In  that  case,  they  would  be  eliminated  in 
this  activity,  because  it  would  not  be  possible  to  develop  a  complete  skill 
definition.  This  activity  is  performed  in  the  following  steps. 


SB/A141:  Select  skill  frames.  This  step  retrieves  from  the  general  ability 
data  base  a  skill  frame  or  template  for  each  general  ability  required  for  learning. 
The  skill  frame  lists  the  kinds  of  information  that  are  required  to  produce  a 
complete  definition  of  a  specific  skill. 


SB/A142:  Incorporate  task-element  objects.  Much  of  the  information 
required  by  the  skill  frames  is  contained  in  the  task-element  descriptions 
associated  with  each  instance  of  a  general  ability.  Specifically,  the  task-element 
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objects  will  be  critical  components  of  a  complete  specific  skill  description.  This 
step  transfers  the  task-element  objects  to  the  appropriate  slots  of  each  skill  frame. 

SB/A143:  Incorporate  domain-specific  knowledge.  Domain  information, 
such  as  the  characteristics  of  displays  and  controls,  and  operating  conditions,  will 
also  provide  a  major  source  of  information  to  complete  the  specific  skill  definitions. 
Some  of  this  information  is  a  part  of  a  task  analysis,  other  information  will  need 
to  be  assessed  directly  from  the  SME.  The  output  of  this  step  will  be  specific  skill 
definitions  that  are  elaborated  to  the  extent  possible  given  aval1  able  information. 

SB/A144:  Select  completed  skill  definitions.  This  step  checks  the  specific 
skill  definitions  for  completeness,  and  eliminates  any  incomplete  definitions. 
Incomplete  skill  definitions  are  an  indication  that  the  general  ability  is  not  really 
required,  but  was  not  eliminated  in  A13.  Consequently,  these  incomplete  frames 
are  not  considered  further  in  the  analysis. 
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SB/A  15:  Determine  Skill  Importance 


This  activity  combines  task  and  task-element  information  to  produce  a 
weight  that  represents  the  relative  importance  of  each  skill. 

SB/A151:  Calculate  raw  score.  This  step  calculates  a  raw  skill  importance 
measure  for  each  specific  skill  from  the  input  values.  The  raw  skill  importance 
weight  apportions  the  task  importance  over  the  skills  that  compose  the  task.  The 
task  importance  is  measured  by  the  time  required  to  train  the  task  to  standard  (or 
total  task  training  time  using  actual  equipment).  The  skill  importance  weight  is 
the  product  of  (a)  the  task  importance  and  (b)  the  ratio  of  task-element  criticality 
to  task  criticality  for  the  task  element  from  which  the  skill  was  generated,  divided 
by  (c)  the  number  of  skills  that  compose  the  task. 

SB/A152:  Determine  sum  over  skills.  This  step  determines  the  total  weight 
for  all  skills.  The  total  weight  is  the  sum  of  the  raw  skill  importance  measure 
over  all  specific  skills.  This  sum  used  in  the  following  step  to  develop  an 
importance  weight  that  sums  to  1.0  over  all  specific  skills. 

SB/A153:  Divide  raw  score  bv  sum.  This  step  divides  the  importance 
weight  for  each  specific  skill  by  the  sum  calculated  in  A152.  The  resulting 
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importance  weight  may  be  then  viewed  as  a  percentage,  since  the  total  weight, 
summed  over  all  specific  skills,  is  1.0. 
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This  activity  selects  skill  training  strategies  for  each  Bkill  identified  in  Al. 
The  skill  training  strategies  specify  the  skills  that  should  be  trained  together,  the 
training  criteria  for  skill  groups,  and  the  training  sequence  for  skill  training.  For 
each  training  strategy,  the  activity  calculates  the  interface  requirements  and  the 
instructor  support  requirements. 

SB/A21:  Group  skills.  This  step  combines  skills  into  groups  that  would  be 
trained  together  using  the  same  training  device.  The  skills  are  grouped  according 
to  the  objects  of  the  skills  and  the  general  ability  involved.  Thus,  skills  in  a  group 
all  involve  the  same  display  or  control  panel,  and  will  involve  a  single  general 
ability,  such  as  pattern  recognition,  or  visualization. 

SB/A22:  Incorporate  training  criterion.  This  step  assigns  a  skill  training 
criterion  to  each  skill  group  identified  in  the  previous  step  (A21).  The  skill 
training  criterion  reflects  the  training  criteria  for  the  tasks  that  are  supported  by 
the  skills  in  a  group,  but  the  characteristics  of  the  skills  may  indicate  that  the 
skills  should  be  trained  to  a  higher  or  lower  standard  than  the  task  performance 
criterion. 


90 


NOK 


SB/A2 


irni: 


Select  Skill  Training  Strategies 


SBOOB 


SB/A23:  Incorporate  training  methods.  This  step  determines  whether  the 
skill  group  requires  or  would  benefit  from  certain  training  methods  that  have 
implications  on  training-device  design.  These  methods  include  part-task  training 
methods  (segmentation,  simplification,  and  fractionation),  procedural  or  cognitive 
pretraining,  or  use  of  augmented  cues  or  feedback.  This  step  does  not  perform  a 
comprehensive  analysis  of  training  method;  rather,  it  concentrates  on  specific 
methods  that  have  known  implications  on  training-device  design.  The  analysis  is 
based  on  research  that  investigated  the  conditions  under  which  Part-Task 
Training  strategies  should  be  recommended  (Knerr,  et  al.,  1986). 


SB/A24:  Determine  interface  requirements.  This  step  determines  interface 
requirements  for  each  skill  group.  Interface  requirements  refer  to  the  ability  to 
provide  appropriate  cues  and  response  feedback  for  skill  acquisition.  Specific 
interface  dimensions  specify  the  characteristics  of  visual  displays,  controls, 
simulation  capabilities,  special  effects,  and  so  forth.  This  activity  determines 
which  interface  dimensions  are  relevant  for  a  skill  group,  and  what  levels  of 
performance  are  required  to  train  the  skill  group  to  the  skill  training  criterion. 
The  output  this  process  is  a  listing  of  the  relevant  interface  dimensions  and  the 
performance  level  on  each  dimension  required  for  training. 


SB/A25:  Determine  instructor  support  requirements.  This  activity 
determines  the  instructional  support  features  that  are  appropriate  for  each  skill 
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group.  The  selected  instructional  support  features  depend  upon  the  skill  entry 
level  performance,  the  skill  training  criterion,  the  skill  sequencing  option  that  is 
selected,  the  general  ability  involved,  and  domain  information.  The  process  may 
be  viewed  as  an  expert  system  that  obtains  information  from  the  task  analysis 
results  and  skill  training  strategy  definitions,  and  supplements  this  information 
with  domain  information  obtained  directly  from  the  SME  when  it  is  required.  The 
rules  will  specify  which  instructional  features  can  improve  training  efficiency  as  a 
function  of  skill  characteristics.  Most  information  required  by  this  activity  will  be 
available  from  task  analysis  or  from  previous  processes.  However,  additional 
information  provided  by  the  SME  will  probably  also  be  required.  The  output  of 
this  activity  is  an  instructional  feature-by*skill  group  matrix  that  indicates  which 
instructional  features  will  enhance  the  training  efficiency  for  each  skill  group. 

The  value  1.0  in  a  cell  of  the  matrix  indicates  that  the  instructional  feature 
indicated  by  the  row  of  the  matrix  can  improve  the  training  efficiency  of  the  skill 
group  indicated  by  the  column  of  the  matrix. 
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SB/A21:  Group  Skills 


This  activity  identifies  groups  of  skills  that  involve  the  same  displays  or 
controls,  and  require  the  same  general  abilities.  The  importance  weight  of  the 
individual  skills  in  a  group  are  combined  to  yield  a  group  weight,  which  is  used  to 
limit  the  number  of  skill  groups  to  the  number  specified  by  the  training  designer. 

SB/A211:  Calculate  obiect-set  least  upper  bound.  The  objects  included  in 
skill  definitions  include  displays,  other  sensory  cues,  controls,  visualizations,  and 
other  memory  representations.  We  assume  that  these  objects  are  arranged  in  a 
hierarchy.  Task  analyses  often  include  a  hierarchical  description  of  displays  and 
controls.  The  requirements  of  this  activity  are  slightly  more  comprehensive.  Each 
object  receives  an  outline  number  that  indicates  it’s  position  in  the  hierarchy.  For 
example,  an  object  that  received  the  number,  23,  is  the  third  minor  branch  from 
the  second  major  branch  from  the  top  node  in  the  hierarchy.  Since  the  objects  are 
related  in  a  hierarchy,  any  collection  of  objects  has  a  unique  least  upper  bound. 
The  least  upper  bound  for  two  objects  is  the  common  beginning  of  their  outline 
numbers.  Thus,  if  two  objects  receive  the  outline  numbers  1234  and  1254,  their 
least  upper  bound  is  the  object  12.  For  example,  a  skill  that  involves  searching  a 
single  display  for  several  items  of  information,  the  objects  would  be  the  display 
and  the  items  of  information.  The  least  upper  bound  would  be  the  display,  since 
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the  information  is  a  direct  des cendent  of  the  display  in  the  hierarchy.  This  step 
determines  the  least  upper  bound  of  the  set  of  objects  for  each  specific  skill. 


SB/A212:  Partition  skills  to  groups.  This  step  partitions  the  set  of  skills 
into  groups  that  have  the  same  object  least  upper  bound  and  involve  the  same 
general  ability.  For  example,  all  skills  that  involve  recognizing  patterns  from  a 
single  display  are  placed  in  the  same  group.  Since  the  skills  are  partitioned,  each 
skill  is  placed  into  one  and  only  one  group.  A  group  may  consist  of  one  or  more 
skills. 


SB/A213:  Calculate  group  weight.  This  step  determines  the  importance 
weight  for  a  group  of  skills  by  adding  the  weights  for  each  skill  within  the  group. 
Since  the  skill  groups  partition  the  set  of  specific  skills,  and  since  the  specific-skill 
weights  sum  to  1.0,  the  group  weights  will  also  sum  to  1.0. 

SB/A214:  Select  too  skill  groups.  This  step  reduces  the  number  of  skill 
groups  being  considered  by  the  analysis  to  a  number  specified  by  the  training 
designer.  First,  the  skill  groups  are  ranked  by  group  weight.  Then,  the  skill 
groups  with  the  greatest  importance  weight  are  chosen  for  further  analysis. 
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SB/A22:  Incorporate  Training  Criterion 

This  activity  specifies  a  training  criterion  for  each  skill  group,  and 
incorporates  the  criterion  into  the  definition  of  the  skill  training  strategy.  The 
first  step  determines  the  maximum  training  criterion  for  the  skill  group, 
depending  on  the  training  criteria  of  the  tasks  supported  by  the  skills  in  the  group 
and  the  characteristics  of  the  skills.  The  second  step  determines  the  extent  to 
which  skill  characteristics  would  favor  skill  training  as  opposed  to  task  training. 
The  third  step  combines  the  results  of  the  first  two  processes  to  recommend  a  skill 
training  criterion,  and  includes  the  criterion  in  the  strategy  definition. 

SB/A221:  Determine  maximum  criterion.  One  way  of  setting  the  criteria 
for  skill  training  is  to  train  each  skill  to  the  task  training  criterion.  Thus,  if  the 
task  requires  performance  at  a  mastery  level,  all  skills  would  be  trained  to  that 
level.  However,  for  some  tasks  it  may  be  optimal  to  train  some  of  the  skills  to  a 
higher  level  than  the  task  performance  criterion.  Training  skills  to  a  higher  level 
will  be  appropriate  for  skills  in  which  responses  are  consistently  mapped  to  cues. 
This  activity  begins  by  determining  from  the  SME  whether  any  of  the  skills  in  a 
skill  group  are  consistently  mapped.  If  some  of  the  skills  are  consistently  mapped, 
then  the  maximum  training  criterion  is  set  to  a  high  level  representing 
automaticity.  If  all  are  variably  mapped,  then  the  maximum  skill-group  training 
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criterion  is  set  to  the  maximum  task  training  criterion  for  the  tasks  supported  by 
the  skill  group. 


SB/A222:  Assess  skill  training  factors.  The  skill  training  criterion  depends 
on  factors  other  than  those  examined  in  the  previous  step  (A221).  For  example, 
factors  such  as  delay  of  feedback,  insufficient  practice  in  the  task  context,  high 
workload,  and  others  (as  described  in  the  previous  section  presenting  the  rationale 
for  skill-based  training),  provide  rationale  for  conducting  skill  training  to  a  higher 
performance  level.  This  activity  obtains  ratings  from  the  SME  of  the  factors  that 
indicate  the  relative  advantages  or  disadvantages  for  skill  training,  and  averages 
these  factors  into  an  index  of  the  values  of  skill  training.  The  index  has  a  range  of 
values  between  0.0  and  1.0,  where  0.0  indicates  the  maximum  advantage  for  task 
training,  0.5  indicates  no  advantage  for  either  skill  training  or  task  training,  and 
1.0  indicates  the  maximum  advantage  of  skill  training. 

SB/A223:  Set  skill  training  criterion.  This  step  calculates  the  skill  training 
criterion  for  a  skill  group  to  be  the  product  of  the  maximum  criterion  calculated  in 
A221  and  the  index  of  the  advantage  of  skill  training  calculated  in  A222.  The 
skill  training  criterion  is  added  to  the  information  that  is  defines  the  skill  training 
strategy. 
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SB/A23:  Incorporate  Training  Methods 


This  activity  determines  whether  the  skill  group  requires  or  would  benefit 
from  certain,  specified  training  methods  that  have  known  implications  on  training- 
device  design.  These  methods  include  part-task  training  methods  (segmentation, 
simplification,  and  fractionation),  procedural  or  cognitive  pretraining,  or  use  of 
augmented  cues  or  feedback.  This  activity  recommends  training  methods  for 
training  individual  skills,  based  on  the  skill  type,  the  entry  skill  level,  the  skill 
difficulty,  and  reasons  for  skill  difficulty.  The  recommendation  is  added  to  the 
information  that  defines  the  training  strategy. 

SB/A231:__  Select  part  or  whole  methods.  This  step  recommends  whether 
skills  in  the  group  should  be  trained  using  part-  or  whole-task  training  methods 
based  on  the  difficulty  of  the  skills  and  the  entry  skill  performance.  If  the  skills 
are  very  difficult  (e.g.,  they  requires  more  than  100  hours  of  training)  and  the 
entry  skill  level  is  low  (the  student  does  not  know  the  skill  at  all),  then  part-task 
methods  are  recommended.  Otherwise,  whole-skill  methods  are  recommended. 

SB/A232:  Recommend  specific  method.  This  step  recommends  specific 
training  methods  (segmentation,  simplification,  procedural  pretraining,  augmented 
feedback,  etc.)  for  each  skill  group  based  on  the  general  ability  involved  and  the 
reasons  that  the  skills  in  the  group  are  difficult.  The  rules  that  make  the 
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required  inferences  are  derived  from  the  training  research  literature.  For 
example,  this  step  will  recommend  a  segmentation  strategy  for  procedural  skills 
that  are  difficult  because  of  their  length. 


SB/A24:  Determine  Interface  Requirements 


This  activity  determines  the  requirements  to  present  cues  and  response 
feedback  for  the  acquisition  of  each  skill  group.  To  determine  interface 
requirements  requires  detailed  knowledge  of  the  task;  much  of  this  detailed 
information  will  be  provided  by  the  SME.  The  output  of  this  activity  is  a  set  of 
requirements  on  several  interface  dimensions.  Both  the  applicable  dimensions 
and  the  required  levels  are  determined  in  the  following  three  steps. 


SB/A241:  Select  relevant  interface  dimensions.  This  step  selects  the 
relevant  interface  dimensions  based  on  the  skill  definition.  A  set  of  rules 
embodies  the  relationships  between  skill  information  and  interface  dimensions. 
For  example,  if  the  skill  requires  visual  pattern  recognition,  interface  dimensions 
that  describe  different  auditory  cues  will  be  irrelevant,  as  will  dimensions  that 
describe  system  controls.  The  output  of  this  task  is  a  set  of  interface  dimensions 
that  will  form  the  basis  of  the  analysis  conducted  in  the  next  two  steps. 


SB/A242:  Perform  preliminary  restrictions.  This  step  eliminates  some  of 
the  levels  of  interface  dimensions  that  represent  either  very  high  levels  or  very 
low  levels,  b  ised  on  information  that  is  available  from  the  task  analyses  or 
previous  analyses,  including  the  skill  group  training  criterion  that  is  a  part  of  the 
skill  training  strategy  and  information  about  the  job  domain.  For  example,  if  the 
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skill  is  performed  on  the  job  with  a  very  simple  display,  then  more  complex 
options  on  that  dimension  may  be  eliminated.  Similarly,  if  the  purpose  of  skill 
training  is  to  familiarize  the  student  with  displays  or  controls,  only  limited 
capability  is  required. 

SB/A243:  Determine  required  levels.  This  step  performs  an  analysis  of  the 
interface  requirements,  based  on  the  detailed  skill  knowledge  of  ihe  SME.  The 
analysis  system  will  ask  the  SME  questions  about  the  cues  required  to  perform 
the  skill  at  a  criterion  level  in  the  job  context.  The  required  levels  on  the  interface 
dimensions  will  depend  on  the  SMEs  answers  to  the  questions.  These  questions 
will  address  the  specific  activities,  performance  cues,  and  response  feedback 
involved  in  skilled  performance.  The  analysis  performed  in  this  step  will  specify 
the  required  level  within  the  range  of  options  determined  in  A242. 
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SB/A3:  Design  Skill-Training  Devices 


This  activity  operates  on  the  skill  groups  that  make  up  selected  skill- 
training  strategies  to  produce  one  or  more  training  device  designs  that  meet  the 
device  requirements  of  the  particular  skills  within  the  resource  constraints.  The 
initial  step  of  this  activity  determines  which  skills  and  tasks  are  associated  with 
each  of  several  general  device  types.  Next,  analyses  are  conducted  to  select  the 
appropriate  instructional  features  and  interface  capabilities  for  each  device.  A 
cost-benefit  analysis  is  conducted  for  each  device  type  to  optimize  training 
enhancement  per  development  dollar.  These  devices  are  then  compared  with 
known  resource  constraints  and  experience  from  allocating  training  (A4),  and 
devices  that  are  not  feasible  are  dropped  from  the  recommendation. 

SB/A31:  Organize  skill  groups  bv  device  requirements.  This  process 
compares  the  device  requirements  of  the  skill  groups  to  the  maximum  capabilities 
of  classes  of  general-purpose  training  device  (e.g.,  microprocessor-based  CBI, 
control  procedures  trainer,  etc.),  and  assigns  the  skill  groups  to  a  specific  class  of 
training  device.  The  general-purpose  devices  are  examined  in  order  of  increasing 
cost  so  that  a  skill  group  will  be  assigned  to  the  least  expensive  class  of  device 
that  has  the  potential  to  provide  the  required  training.  Skill  groups  that  require 
features  that  are  not  available  in  any  of  the  general  classes  of  training  device  are 
candidates  for  the  development  of  a  specialized  training  device  and  may  be  treated 
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singly,  or  as  a  whole  in  the  following  activities.  The  result  is  a  iwo-ouneiwtuuai 
matrix  of  device  types  (including  a  specialized  device)  by  skill  group  that  reflects 

the  assignment. 


SB/A32:  Select  instructional  features  bv  device.  This  activity  determines 
the  instructional  features  that  are  appropriate  to  train  the  skills  assigned  to  each 
training  device,  based  upon  the  instructor  support  requirements  of  the  skills  to  be 
trained  and  the  capabilities  of  the  class  of  training  device  being  considered.  The 
subactivities  of  this  activity  determine  which  instructional  features  have  the 
greatest  expected  impact  on  training  efficiency,  given  their  cost.  The  activity 
produces  a  list  of  the  instructional  features,  ordered  by  decreasing  benefit/coBt 
ratio.  The  designer  may  eliminate  the  instructional  features  from  this  list  that  do 
not  provide  adequate  value,  or  that  exceed  resource  constraints. 

SB/A33:  Select  interface  features  bv  device.  This  activity  determines  the 
computational  and  interface  capabilities  that  are  appropriate  to  train  the  skills 
assigned  to  each  training  device,  based  upon  the  interface  requirements  of  the 
skills  to  be  trained  and  the  capabilities  of  the  class  of  training  device  being 
considered.  The  first  subactivity  restricts  the  options  considered  to  be  consistent 
with  the  interface  requirements,  maximum  device  capabilities,  and  user  desires. 
The  second  subactivity  calculates  the  cost  and  benefit  of  interface  options.  The 
third  subactivity  computes  the  optimal  device  designs  according  to  the  ratio  ot 
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benefit  to  cost.  The  activity  produces  a  list  of  the  computational  and  interface 
features,  ordered  by  decreasing  benefit/cost  ratio.  The  designer  may  eliminate 
features  from  this  list  that  do  not  provide  adequate  value,  or  that  exceed  resource 
constraints. 


SB/A34:  Recommend  device  designs.  This  activity  performs  a  cost-benefit 
analysis  to  balance  the  level  of  computational  and  interface  capabilities  with  the 
level  of  instructional  features.  In  determining  the  overall  benefit,  the  benefit  for 
instructor  support  features  is  divided  by  its  maximum  value  to  put  it  in  the  range, 
[0,1].  Then  the  overall  benefit  due  to  interface  features  may  be  weighted  equally 
to  the  overall  benefit  due  to  instructor  support  features,  because  both  benefits  are 
measured  on  a  scale  in  which  1.0  signifies  that  the  option  meets  all  skill  group 
requirements.  However,  the  designer  has  the  freedom  to  modify  these  weights 
based  upon  experiences  from  allocating  training  (A4).  The  cost  in  this  analysis  is 
the  development  cost  of  the  devices.  The  relative  benefit  of  the  interface 
capabilities  and  instructor  support  features  are  part  of  the  designs  determined  in 
A32  and  A33.  The  designer  may  obtain  a  recommended  design  by  specifying  one 
of  the  following  three  types  of  constraints:  (a)  The  designer  may  specify  the  target 
cost.  The  system  will  then  identify  the  device  design  (i.e.,  the  combination  of 
interface  and  instructor  support  features)  that  provides  the  greatest  benefit  at 
that  cost  or  less,  (b)  The  designer  may  specify  the  target  benefit.  The  system  will 
identify  the  device  design  that  provides  at  least  the  target  benefit  at  the  lowest 
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cost,  (c)  The  designer  may  specify  a  minimum  configuration,  that  is,  a  set  of 
features  that  must  be  included  in  the  design.  The  system  will  then  identify  a 
system  design  that  includes  all  required  features  and  those  features  that  are  more 
efficient  (based  on  benefit/cost  ratio).  The  selection  is  made  by  selecting  interface 
or  instructor  support  features  in  order  of  benefit/cost  ratio  until  one  of  the 
constraints  is  met.  The  recommended  device  design  is  forwarded  to  A4  for 
consideration  in  allocating  training  resources. 


105 


ns EOT 


S/m*  Rfi 
PKUfCT:  SdbOM 

WfTB.  I  2  J  4  S  I  7  I  I  j> 


"SfitnTETK- 

•Dft 


T| 

til  " 


rm _ 

xfiraw 
titar 


n  a* TOT: 


Cl 

Device 
Requirement*  l 


Skill  Croups 


Training  Device 
Infornwlion 


Orgonl:* 
SkEl  Croups 
by  Device 
Rgm/its.  1 


Instructor 

Support 

Requirement* 


Skfll/Devlc* 

Matrix 


12 


Device 

Options 


Instructor 
Support  Options 


Inlorfocs  Options 


Interlace 

Requirement* 


Selected 
I  Instructional 
ulructtood  I  Feature* 
Feature*  by 
~  ■  Cat  : 


C3 


Rtsourcs 

Comtralnt* 


Sol  set  kiter- 
Iocs  Featured  ttOT 
by  Orrles  J  ^ 

Satogory  3| 


Selected 

Interface 

Feature* 


S80I5 


C4 

Previous 
AJ  location 
Experience 


Recommend 

Device 

Design* 


Designer] 


Training 

Device 

Designs 


1 


01 


N00C 


SB/A3 


rmi: 


Design  Skill-Training  Devices 


SB013  j 


SB/A32:  Select  Instructional  Features  bv  Device  Category 


This  activity  identifies,  given  a  development  cost,  the  instructional  features 
that  will  improve  training  for  the  most  skills.  Instructional  features  are  presumed 
to  influence  the  efficiency  of  training  rather,  represented  by  the  time  multiplier  of 
the  learning  model,  than  the  maximum  effect  of  training  or  transfer  of  training, 
represented  by  the  asymptote.  Five  major  activities  compose  this  tool:  An 
effectiveness  score  is  calculated  for  each  instructional  feature;  the  effectiveness 
6core  is  converted  to  a  feature  benefit  measure;  a  benefit/cost  ratio  is  calculated; 
instructional  features  are  sorted  by  benefit/cost  ratio;  and  optimal  combinations  of 
features  are  listed.  The  output  of  this  activity  is  a  list  of  feasible  instructional 
features  ordered  by  benefit/cost  ratio. 

SB/A321:  Calculate  effectiveness  score.  This  activity  determines  the 
effectiveness  score  by  finding  for  each  instructional  feature  the  sum  of  the  skill- 
group  weights  for  each  skill  group  that  requires  that  instructional  feature.  The 
instructional  features  considered  in  this  analysis  are  limited  by  those  that  are 
consistent  with  the  capabilities  of  general  training  device  type  being  specified. 

SB/A322:  Calculate  feature  benefit.  This  analysis  broadens  the  concept  of 
instructional  feature  effectiveness  to  incorporate  the  likelihood  that  the  feature 
will  be  used,  expressed  as  a  probability,  based  on  historical  usage  data.  The 
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instructional  feature  benefit  is  given  by  the  product  of  the  effectiveness  score 
calculated  in  A321  and  an  instructional  feature  weight  that  reflects  the  relative 
usage  probability  for  that  feature.  The  feature  benefit  may  be  viewed  as  the 
expected  proportion  of  skill  groups  addressed  by  an  instructional  feature. 

SB/A323:  Compute  benefit/cost  ratios  for  options.  This  process  combines 
the  calculated  instructional  feature  benefits  with  the  assessed  feature  costs,  by 
dividing  the  benefit  by  the  cost  to  obtain  a  benefit/cost  ratio. 

SB/A324:  Sort  options  bv  benefit/cost  ratios.  This  activity  orders  the 
instructional  features  according  to  decreasing  benefit/cost  ratio.  If  the  features 
are  independent,  this  order  represents  the  optimal  order  in  which  to  incorporate 
instructional  features  into  a  training-device  design  as  a  function  of  cost. 


SB/A325:  List  optimal  features.  This  activity  presents  the  results  of  the 
analysis  so  that  the  user  may  determine  the  instructional  features  that  should  be 
included  to  meet  cost  or  benefit  criteria.  The  optimal  list  of  features  at  any  cost 
(or  benefit)  level  is  found  by  adding  features  to  the  list  in  sorted  order  until  the 
cumulative  cost  (or  benefit)  is  equal  to  the  criterion  level.  This  activity  takes  the 
range  of  costs  and  produces  as  output  the  set  of  optimal  instructional  feature 
packages  for  all  costs  in  the  range.  Included  in  the  output  are  the  cost  and  benefit 
of  each  package. 
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SB/A33:  Select  Interface  Features  by  Device  Category 


This  activity  consolidates  skill  interface  requirements  for  the  skill  groups 
being  considered  for  a  training  device  with  the  computational  and  interface 
capabilities  of  the  class  of  training  device  being  designed  to  produce  a  design  that 
meets  the  most  critical  requirements  at  the  minimum  cost.  Computational  and 
interface  capabilities  are  assumed  to  influence  the  maximum  effect  of  training  or 
transfer  of  training  rather  than  the  speed  of  learning.  The  output  of  this  activity 
is  a  list  of  feasible  interface  options,  ordered  by  benefit/cost  ratio. 

SB/A331:  Construct  training  device  options.  This  activity  selects  a  set  of 
active  interface  options  to  be  considered  in  the  analysis  from  the  set  of  all  options. 
If  a  general-purpose  training  device  is  being  specified,  then  the  active  options  are 
the  components  of  the  training  device  for  which  options  exist.  For  example,  for  a 
microprocessor-based  CBI  system,  these  options  would  include  the  display 
controller,  the  display  monitor,  video  capability,  the  processor  capabilities,  and  so 
forth.  If  a  special-purpose  training  device  is  being  specified,  then  the  options  will 
be  determined  by  the  interface  dimensions  on  which  the  skill  groups  being 
considered  have  interface  requirements  (from  A243).  The  levels  that  will  be 
included  range  from  no  capability  to  the  maximum  requirement  on  each 
dimension.  The  user  may  modify  this  choice  by  selecting  or  eliminating  interface 
dimensions  and  specifying  the  levels  of  capability  on  each  dimension  that  will  be 
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evaluated  by  the  analysis.  The  output  of  this  activity  is  a  set  of  candidate 
interface  dimensions  and  levels,  with  associated  technical  performance  and  cost 
information  that  are  drawn  from  the  data  base. 


SB/A332:  Calculate  cost  and  benefit  of  options.  This  activity  consists  of 
four  steps  that  establish  development  costs  and  training  benefits  for  each  option. 
The  costs  and  benefits  are  comparable  across  interface  dimensions  and  are  based 
on  the  technical  performance  of  the  options.  Costs  are  determined  from  technical 
performance  by  an  estimation  function  in  the  first  subactivity.  The  remaining 
three  subactivities  determine  benefit  by  determining  skill  trainability,  calculating 
a  benefit  score  within  each  interface  dimension,  and  determining  weights  that 
place  benefit  on  a  common  scale  across  dimensions. 

SB/A333:  Compute  optimal  device  designs.  This  process  has  three 
operations.  The  optimal  training-device  designs  are  based  upon  the  incremental 
benefit/cost  ratios  of  the  options.  After  these  ratios  are  computed,  the  options  are 
sorted  in  priority  order,  and  optimal  designs  are  defined  for  user-specified  cost  or 
performance  levels.  The  output  of  this  activity  is  a  set  of  optimal  interface  designs 
at  various  cost  levels.  Included  in  the  output  are  the  cost  and  benefit  of  each 
design. 
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This  process  yields  the  benefit  and  cost  data  needed  to  compute  the 
incremental  benefit-to-cost  ratios  for  the  optimization  in  SB/A333. 

SB/A3321:  Calculate  costs.  This  activity  addresses  the  development  costs 
of  a  training  device  and  is  a  straightforward  calculation  from  interface  dimension 
data  using  the  cost  equations  shown  below.  The  cost  estimation  function  is 
designed  so  that  constant  increments  of  technical  performance  are  more  expensive 
at  a  high  performance  level  than  they  are  at  a  low  performance  level.  The  cost 
equations  relate  the  development  costs  to  the  technical  performance  index  in  the 
range  of  options  under  consideration,  as  follows: 

CDEVjffi(TPJm)  =  CMINj  +  (CMAXj  -  CMINp  *  CREL^ 

where 

CDEVjm  =  the  development  cost  of  training  device  option  m  on  interface 
dimension  j, 

CMINj  =  the  development  cost  of  the  least  capable  training  device  option  on 
interface  dimension  j, 
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CMAXj  =  the  development  cost  of  the  most  capable  training  device  option  on 
interface  dimension  j, 

CREL,m  =  the  normalized  development  cost  of  training  device  option  m  on 
interface  dimension  j,  and 

TPjm  =  the  technical  performance  index  of  training  device  option  m  on 
interface  dimension  j. 

CRELjm  is  calculated  from  the  relative  technical  performance  (TPRELjm) 
which  measures  the  technical  sophistication  of  each  option  m  on  interface 
dimension  j.  TPREL,m  is  also  defined  on  a  scale  from  0  to  1.0  as  a  function  of  the 
absolute  technical  performance  (TP),  and  is  calculated  as  follows: 

TPjm  -  TPMINj 

TPRELjm(TPjm)  =  _ 

TPMAXj  -  TPMINj 


where 

TPMAXj  =  the  technical  performance  of  the  most  capable  option  on  interface 
dimension  j, 

TPMINj  =  the  technical  performance  of  the  lest  capable  option  on  interface 
dimension  j,  and 
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TPREL,m  =  the  normalized  (between  0  and  1.0)  technical  performance  of 
training  device  option  m  on  interface  dimension  j. 

TPjml  TPMINj,  and  TPMAXj  are  all  part  of  the  existing  data  base  on  training 
device  options. 


used: 


To  compute  the  normalized  development  cost,  the  following  equation  is 


CRELjm  =  -(1/Ep  ln[l  -  TPRELjm  (1  -  expCE*))] 


where 


Ej  =  a  constant  that  describes  the  interface  dimension. 

This  function  has  a  property  that  technology  becomes  more  costly  as  the  constant 
exponent  (Ej)  increases  in  size. 

SB/A3322:  Determine  skill  trainabilitv.  This  is  the  first  step  in  the 
calculation  of  benefits  for  training  device  options.  This  activity  examines  each 
option  of  each  interface  dimension,  and  compares  the  technical  performance  of 
that  option,  TPjm,  with  the  corresponding  interface  requirements  of  each  skill 


112 


group  (as  defined  in  A243)  being  addressed  by  the  training  device.  The  activity 
records  via  a  three-dimensional  skill  trainability  matrix,  TRNs,ro,  which  skill 
groups  have  requirements  that  can  be  met  by  each  level  of  each  interface 
dimension.  The  output  of  this  process  is  the  three-dimensional  matrix  (skill 
groups  by  interface  dimension  by  level)  that  enumerates  the  skill  group 
requirements  that  are  met  by  each  interface  level. 


SB/A3323:  Calculate  benefit  scores.  This  process  uses  the  skill  trainability 
matrix,  TRNSjm,  generated  in  the  previous  process  (A3322)  and  the  skill  group 
importance  weights  calculated  in  A2 13  to  calculate  the  benefit  score  of  each 
option.  The  benefit  score  is  defined  to  represent  the  relative  benefit  of  an  option 
within  its  interface  dimension.  For  all  options  within  an  interface  dimension,  the 
benefit  scores  represent  the  weighted  ratio  of  skills  that  can  be  trained  within  that 
interface  dimension.  These  benefit  scores  fall  between  0  and  1.0;  where  0 
represents  the  inability  to  train  any  skills  and  1.0  represents  the  ability  to  train 
all  skills  within  that  dimension. 


The  benefit  scores  of  each  interface  dimension  level  are  calculated  by 
summing  the  skill-group  weights  for  all  skill  groups  that  have  requirements  that 
are  met  by  the  specific  level.  That  is,  skill-group  weights  are  placed  in  the 
skill-group  trainability  matrix  for  all  cells  in  which  the  skill  group  was  trainable. 
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The  resulting  matrix  is  then  summed  over  skill  groups  to  produce  a 
two-dimensional  (interface  dimension  by  level)  benefit  matrix. 


□ 


01 


SB/A3324:  Calculate  benefit  weights.  The  interface  dimensions  receive  a 
weight  that  reflects  the  extent  to  which  important  skill  groups  require  high 
technical  performance  on  an  interface  dimension.  The  interface  dimensior 
weights  place  benefit  on  a  commrn  scale  across  interface  dimensions.  Each  of  the 
skill-group  requirements  is  multiplied  by  the  corresponding  skill-group  weight, 
and  the  resulting  values  (which  are  two-dimensional,  skill  group  by  interface 
dimension  matrix)  are  summed  across  skill  groups.  These  interface  benefit 
weights  are  then  normalized  to  sum  to  1.0  to  maintain  scale  relationships  among 
benefit  values. 
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SB/A333:  Compute  Optimal  Device  Designs 

This  process  assembles  the  device  interface-dimension  levels  based  upon  the 
incremental  benefit-to-cost  ratios  to  attain  these  levels.  It  uses  this  information  to 
determine  the  optimal  training  device  design  at  any  user-specified  level  of 
development  cost  or  benefit.  These  designs  may  be  evaluated  further  by  using 
them  as  inputs  to  the  allocate  training  activity  to  determine  which  option  meets 
the  training  requirements  at  the  least  cost. 


SB/A3331:  Compute  benefit/cost  ratios  for  options.  This  activity  u<=os  the 
cost  and  benefit  data  from  the  previous  major  process  (A332).  The  weighted 
incremental  benefit/cost  ratio  of  a  level  on  an  interface  dimension  is  calculated  by 
comparing  the  costs  and  benefits  of  that  level  to  the  costs  and  benefits  of  the 
previous  level  (levels  are  ordered  by  technical  performance).  Specifically,  the 
incremental  benefit/cost  ratio  is  obtained  by  dividing  the  incremental  benefit  by 
the  incremental  cost.  This  quotient  is  then  multiplied  by  the  appropriate  benefit 
weight  to  obtain  a  figure  that  is  comparable  across  dimensions. 


Before  the  options  can  be  sorted  by  their  benefit-to-cost  ratios,  these  ratios 
must  be  guaranteed  to  be  decreasing  within  each  interface  dimension.  This  is 
done  by  dropping  any  level  from  consideration  that  is  not  cost-efficient  within  that 
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dimension.  When  only  cost-efficient  options  remain,  the  incremental  benefit/cost 
ratios  are  recalculated  from  the  remaining  options. 


SB/A3332:  Sort  options  bv  benefit/cost  ratios.  Because  the  interface 
dimensions  have  been  constructed  to  be  independent  building  blocks  of  a  training 
device  (to  the  extent  that  this  is  possible),  they  can  be  prioritized  solely  on  the 
basis  of  the  incremental  benefit/cost  ratio.  The  output  of  this  activity  lists  the 
options  on  all  interface  dimensions  in  order  of  decreasing  benefit/cost  ratio. 

SB/A3333:  Compute  device  designs.  This  activity  presents  the  results  of 
the  analysis  so  that  the  user  may  determine  the  interface  options  that  should  be 
included  to  meet  cost  or  benefit  criteria.  The  optimal  list  of  features  at  any  cost 
(or  benefit)  level  is  found  by  adding  options  to  the  list  in  sorted  order  until  the 
cumulative  cost  (or  benefit)  is  equal  to  the  criterion  level.  Thi6  activity  takes  the 
range  of  costs  and  produces  as  output  the  set  of  optimal  interface  feature  packages 
for  all  costs  in  the  range.  Included  in  the  output  are  the  cost  and  benefit  of  each 
package. 
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SB/A4:  Allocate  Training 


This  activity  selects  the  previously  defined  STSs  and  associated  devices  that 
minimize  the  total  cost  of  student  training.  Cost  is  minimized  using  a  sequential 
optimization  algorithm.  At  each  stage  of  the  algorithm  a  payoff  matrix  is 
constructed  showing  the  costs  to  execute  the  STSs  on  various  devices  and  the  cost 
savings  that  will  be  realized  in  task  training  by  prior  skill  training.  The 
difference  between  task  training  cost  savings  and  STS/device  training  costs  is  the 
total  savings  for  that  STS/device  combination.  The  STS/device  yielding  the 
greatest  total  savings  is  selected  as  the  next  STS/device  to  be  included  in  skill 
training.  When  no  STS/device  remains  offering  greater  savings  than  costs,  the 
allocation  is  completed. 


SB/A41:  Detail  skill  interaction  with  task  training.  In  order  to  determine 
the  impact  of  skill  training  on  subsequent  task  training,  it  is  necessary  to  address 
the  complete  student  training  program,  encompassing  the  tasks  to  be  trained,  the 
devices  to  be  employed,  and  the  learning  curves  versus  cost  expenditures  for  these 
devices.  In  this  activity,  these  factors  are  used  to  construct  a  skill  impact 
evaluation  framework  to  be  employed  later  in  the  optimization.  The  precise 
nature  of  this  framework  will  depend  on  the  form  and  quality  of  skill-to-task 
transfer  research  data. 
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SB/A42:  Determine  per-student  STS/device  costs.  OSBATS  device  costing 
methodology  is  used  to  determine  per-student  costs  of  the  candidate  STSs  on 
applicable  devices.  Skill  learning  curves  are  constructed  for  each  device  based  on 
interface  and  instructional  features.  The  training  time  to  bring  the  student  to 
criterion  for  the  skills  addressed  by  the  STS  is  determined  for  each  STS/device 
combination.  This  training  time  is  used  as  the  basis  for  STS/device  costing.  The 
per-student  cost  is  based  on  the  development  cost  of  the  device  plus  the  fixed  and 
variable  costs  associated  with  training  the  student  for  the  required  training  time. 


SB/A43:  Determine  impact  on  task  training  costs.  Task  entry  performance 
is  a  function  of  the  skill  levels  attained  prior  to  task  training,  and  on  tasks  trained 
earlier  in  the  student  training  program.  In  addition  to  effecting  increases  in  task 
entry  level  performance,  skill  training  may  cause  an  acceleration  in  the  task 
learning  rate.  Depending  on  the  precise  skill  impact  evaluation  framework 
employed,  these  benefits  will  be  considered  explicitly  or  implicitly. 

SB/A44:  Choose  STS/device  for  greatest  payoff.  The  total  payoff  for  a 
candidate  STS/device  combination  is  the  total  cost  savings  for  all  the  tasks  to  be 
trained,  less  the  cost  of  the  STS/device  training.  These  payoffs  are  computed  for 
each  applicable  STS/device  combination,  and  that  STS/device  with  the  greatest 
payoff  is  included  in  pre-task  skill  training.  If  no  STS/device  yields  a  positive 
payoff,  then  the  training  allocation  process  is  concluded. 
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This  activity  consists  of  two  steps:  first,  characterize  the  optimal  task 
training  program  without  skill-based  training.  Second,  structure  the  way  in  which 
this  training  program  will  be  impacted  by  improved  initial  student  skills. 

SB/A411:  Characterize  baseline  training  and  costs.  Task  learning  curves  for 
existing  task  training  devices  may  be  available  from  empirical  research,  while 
those  for  yet-to-be  fielded  training  devices  may  be  estimated  using  the  learning 
curve  construction  methodology  developed  for  OSBATS.  This  latter  methodology 
employs  indexes  of  acceptable  device  fidelity,  elicited  from  SMEs  for  each  task. 
Learning  curves  and  transfer  functions  are  employed  to  estimate  learning  rates 
and  asymptotes  on  training  devices  with  greater  or  lesser  fidelities,  and  with 
selected  instructional  features.  (See  Sticha,  et.  al.,  1988.) 

The  sequencing  of  task  training  and  training  device  employment  may  be 
specified  by  the  user,  or  may  be  optimized  using  the  OSBATS  training  device 
selection  methodology  (Sticha,  et.  al.,  1988).  The  prescribed  sequence  of  tasks  and 
devices,  and  the  prescribed  performance  levels  at  which  device  transitions  are 
made  is  then  "canned"  for  efficient  further  reference.  The  level  of  detail  saved  will 
depend  on  the  precise  nature  of  the  skill  impact  evaluation  framework.  At  the  one 
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extreme,  only  the  training  times  and  associated  costs  would  need  to  be  saved, 
whereas,  at  the  other  extreme,  entire  learning  curve  formulas  would  be  saved. 


SB/A412:  Characterize  skill  impact  on  training.  The  way  in  which  skill 
impact  on  task  training  is  assessed  will  depend  on  the  skill  impact  evaluation 
framework  selected.  That  framework  will  be  tailored  to  the  nature  and  quality  of 
skill-to-task  research  data  and  the  nature  of  SME  involvement.  The  framework 
will,  in  any  case,  require  the  assignment  of  parameters,  e.g.,  task  learning  curve 
parameters,  or  skill/task  importance  weights.  In  this  activity  those  parameters 
will  be  quantified  based  on  information  developed  in  SB/A411. 
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SB/A42:  Determine  Per-Student  STS/Device  Costs 


To  each  STS  are  associated  criterion  levels  for  certain  skills.  To  achieve 
these  criterion  levels,  different  amounts  of  training  will  be  required,  depending  on 
the  device  employed.  For  each  applicable  STS/device  combination  these  training 
times  are  determined  and  the  associated  cost  of  device  employment  calculated. 
Although  fairly  informal  estimation  methods  may  be  employed,  a  more  formal 
procedure,  as  outlined  below,  is  preferred. 

SB/A421:  Determine  skill  learning  curves  for  device.  Skill  learning  curves 
are  constructed  for  each  applicable  device,  using  empirical  research  results  or 
OSBATS  methodology.  The  latter  methodology  relies  on  SME  judgments  to 
provide  fidelity  requirements  for  various  levels  of  skill  acquisition,  and  then 
employs  a  mathematical  model  to  estimate  learning  rates  for  the  training  device, 
based  on  the  fidelity  levels  and  instructional  features  of  that  device.  The  resulting 
performance  function  PCtjs.m.a^c.k)  expresses  the  skill  level  achieved  after 
training  for  time  t.  The  function  involves  parameters  s,  m  a,,  c,  and  k,  identified 
as  below.  In  OSBATS,  the  learning  curve  was  formulated  as 

P(t)  =  aT  {1  ■  [1  +  mc(s+t)3'k  }. 
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k  determines  the  basic  learning  curve  shape,  m  is  a  time  multiplier,  effecting  a 
compression  or  expansion  of  the  learning  curve  along  the  time  axis,  c  is  simply  a 
time  scaling  constant.  The  transfer  ratio,  ay,  is  a  function  of  the  medium  fidelity 
levels  {rj  in  each  of  the  different  fidelity  dimensions,  i.  The  OSBATS  methodology 
for  determining  also  employs  SME  estimates  of  required  fidelity  levels  {R,}.  In 
addition,  an  adjustment  constant  x  is  employed  to  calibrate  the  SME’s  estimates. 
The  algorithm  for  determining  ay  may  also  involve  constants  {f*}  relating  the  limits 
to  fractional  degradation  in  transfer  do  to  poor  device  fidelity,  s  is  an  effective 
head  6tart  time,  selected  so  that  P(0)  corresponds  to  actual  student  entry 
performance.  For  the  complete  procedure  for  relating  these  various  parameters, 
see  Sticha,  et.  al.,  1988. 


SB/A4222:  Determine  required  STS  training  time  bv  device.  Once  the  6kill 
learning  curves  have  been  constructed,  the  time  required  for  STS  training  is 
determined  as  the  time  required  to  move  from  entry  skill  level  to  the  criterion 
skill  levels  6et  for  the  STS  in  question. 

SB/A4223:  Determine  per-student  cost  bv  STS  and  device.  The  per-student 
cost  rate  for  a  particular  device  is  determined  from  device  development  and 
operational  cost  data,  together  with  knowledge  of  the  demand  for  that  device,  a 
function  of  the  number  of  students  to  use  it  annually,  and  the  amount  of  time 
each  student  is  to  be  trained  on  it.  The  actual  device  costing  methodology  will  be 
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similar  to  that  adapted  by  OSBATS  (Sticha,  et.al.,  1988).  Typically,  the  cost  curve 
will  be  linear  with  slope  B  =  (I/L  +  F)/U  +  r,  where  I  represents  per  device 
investment  cost  (i.e.,  total  investment  cost  assessed  across  all  training  devices 
procured),  L  =  life  cycle,  F  =  fixed  annual  operations  cost  (per  device),  U  =  annual 
device  utilization  in  student-hours,  and  r  a  variable  operations  cost,  per  student 
hour.  These  parameters  are  estimated  using  standard  costing  procedures. 
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This  activity  estimates  the  benefit  of  adding  a  particular  candidate 
STS/device  combination  to  those  already  selected  for  inclusion.  The  evaluation 
consists  of  three  steps.  The  first  is  to  determine  the  cost  of  training  all  tasks  to 
criterion  without  skill  training  with  the  candidate  STS/device  combination  (but 
with  skill  training  on  any  STS/device  combinations  already  selected  for  inclusion 
in  earlier  iterations  of  the  allocation  algorithm).  The  second  step  is  to  determine 
the  cost  of  training  if  the  candidate  STS/device  is  included.  The  third  step  is  to 
simply  calculate  the  arithmetic  difference. 


SB/A431:  Determine  training  cost  w/o  STS/device.  On  the  first  iteration 
through  the  skill  training  allocation  algorithm,  the  cost  of  training  without  the 
candidate  STS/device  combination  is  just  the  baseline  training  cost  estimate 
developed  in  SB/A411.  On  subsequent  iterations  of  the  skill  training  allocation 
algorithm,  the  cost  will  be  saved  from  the  preceding  visit  to  SB/A44,  and  will 
included  the  per-student  cost  of  skill  training  on  previously  selected  STS/device 
combinations,  as  well  as  the  cost  of  subsequent  task  training.  (On  each  iteration 
of  the  algorithm,  the  total  cost  of  skill  training  is  increased,  but  is  more  than 
offset  by  reductions  in  the  total  cost  of  task  training). 
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SB/A432:  Determine  training  cost  with  STS/device.  The  skill  impact 
evaluation  framework  developed  in  SB/A41  is  employed  at  this  step  to  estimate 
the  reduced  cost  of  task  training  due  to  time  compression,  generalization,  and 
reduction  of  attention  requirements  for  each  task  as  a  result  of  improved  entry 
skills  associated  with  the  candidate  STS/device.  The  cost  of  previously  selected 
STS/devices  is  included  in  this  estimate,  but  the  cost  of  the  candidate  STS/device 
is  excluded.  The  impact  of  previously  selected  STS/devices  on  learning  curves  is 
taken  into  account  in  this  activity,  through  revised  pre-task  training  entry  skill 
levels. 

SB/A433:  Determine  training  cost  savings  with  STS/device.  The  net  impact 
that  the  candidate  STS/device  would  have  on  per-student  task  training  costs  is 
just  computed  as  the  difference  between  the  training  cost  determined  in  SB/A431 
and  that  determined  in  SB/A432. 
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SB/A44:  Choose  STS/Device  for  Greatest  Payoff. 


At  each  stage  of  the  algorithm  a  payoff  matrix  is  constructed  and  used  to 
determine,  for  each  remaining  candidate  STS/device,  the  task  training  cost  savings 
less  the  STS/device  employment  cost.  The  selection  of  the  most  effective 
STS/device  is  then  a  matter  of  identifying  that  one  yielding  the  greatest  payoff. 


SB/A441:  Construct  payoff  matrix.  A  Skill  Training  Payoff  Matrix  is 
constructed  by  arraying  the  STS/device  per-student  costs  against  the  per-student 
task  training  savings 
determined  in  the  preceding 
activities.  An  example  of  the 
basic  payoff  matrix  is  shown  at 
the  right. 


STS/ 

Per- 

Task 

Training  Cost 

Savings 

Device 

Student 

for 

Task 

Combo 

Cost 

1 

2  3  . 

.  .  .  n 

1 

$370 

$100 

$50  $0  . 

.  ..  $20 

2 

$1200 

$300 

e 

$800 

9 

$700 

SB/A442:  Determine  payoff  for  each  STS/device.  The  information  in  the 
basic  payoff  matrix  lends  itself  to  determination  of  a  total  per-student  task 
training  cost  savings  realized  for  each  STS/device  combination.  This  savings  is 
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just  the  sum  across  all  tasks  (for  the  job  specialty  of  the  persons  in  question).  The 
net  payoff  for  each  STS/Device  is  then  computed  as  the  total  per-student  task 
training  savings  less  the  per-student  cost  of  the  STS/device,  if  employed.  This 
gives  rise  to  an  expanded  payoff  matrix,  as  shown  below. 


STS/  Per-  Task  Training  Cost  Savings  Per-  Payoff 

Device  Student  for  Task  Student  -Savings- 

Combo  Cost _ 1 _ 2 _ 3  ....  n  Savings _ Cost 

1  $370  $100  $50  $0  ....  $20  $970  $600 

2  $1200  $300  .  $1450  $250 

8  $800  .  $2000  $1200 

9  $700  .  $400  -$300 


SB/A443:  Select  STS/device  with  greatest  payoff.  The  next  STS/device  that 
should  be  selected  is  simply  that  one  yielding  the  greatest  positive  payoff,  e.g., 
STS/device  #8  in  the  example.  Revised  pre-task  skill  levels  and  baseline  training 
costs  are  then  updated  and  used  in  the  next  iteration  of  the  training  allocation 
algorithm.  At  some  iteration,  the  payoffs  of  all  remaining,  unselected  STS/device 
combinations  may  be  negative,  indicating  that  all  cost  effective  STS/devices  have 
been  selected  and  removed  from  the  candidate  list.  At  that  point  the  allocation 
algorithm  is  terminated. 


129 


DISCUSSION 


Skill  training  as  a  distinct  part  of  instruction  in  complex  domains  has  only 
been  attempted  in  limited  applications  in  which  the  need  is  clear,  not  as  a 
generally  prescribed  curriculum  component.  One  primary  reason  that  skill 
training  has  not  become  a  routine  part  of  the  training  process  is  the  difficulty  of 
defining  and  isolating  skills.  Another  is  the  difficulty  of  developing  skill  training 
strategies,  and  knowing  when  skill-based  training  is  more  efficient  than  standard 
training.  There  is  also  the  difficulty  of  specifying  the  conditions  in  which  skill 
training  will  improve  mission  performance.  Underlying  these  difficulties  are  the 
considerable  limitations  of  our  knowledge  of  the  interactions  between  skill  and 
task  training  options,  on  the  one  hand,  and  skilled  mission  performance,  on  the 
other. 


It  is  difficult  to  separate  the  skills  from  the  tasks  in  which  they  are 
required,  but  the  procedures  we  have  developed  indicate  that  it  can  be  done. 
However,  the  general  applicability  of  these  procedures  has  not  been  determined. 
Their  refinement  will  require  application  of  the  methods  to  a  variety  of  problems. 
We  need  to  apply  the  skill-identification  and  strategy-selection  methodology  to 
several  domains  to  get  more  systematic  and  comprehensive  procedures  that  can  be 
applied  confidently  to  a  variety  of  job  domains  by  training  designers  and  SMEs. 

Two  concerns  motivated  this  work:  (a)  how  to  produce  better  trained 
personnel  more  efficiently,  and  (b)  how  to  provide  the  the  most  effective  training 
for  the  least  cost.  While  these  concerns  seem  to  be  two  sides  of  the  same  coin,  the 
issues  they  raise  are  rooted  in  different  fields  of  study.  The  first  question  leads 
one  to  look  for  instructional  principles  on  which  training  can  be  based.  It  focuses 
attention  on  the  psychological  bases  of  performance  and  investigates  methods  that 
exploit  our  understanding  of  those  psychological  processes.  The  second  question 
looks  at  how  instructional  methods,  having  been  selected  to  solve  a  particular 
training  problem,  can  be  effectively  implemented  within  prevailing  technological 
and  cost  constraints. 

Up  to  now  research  on  skill-based  training  has  been,  with  few  exceptions, 
driven  by  one  or  the  other  concern,  but  not  both.  Instructional  research  that  has 
the  stated  goal  of  developing  'low-cost"  trainers  have  specified  the  cost  constraints 
only  at  the  most  general  level  (e.g.,  develop  a  PC-based  trainer).  The  specific 
features  of  the  system  seem  to  fall  out  of  the  narrowly  defined  research  goals  on  a 
particular  task,  rather  than  any  systematic  process.  Training-system  development 
projects  tend  to  be  driven  either  by  perceived  benefits  of  enhanced  technology  or 
cost  limitations.  Clearly,  an  optimal  training-system  development  process  needs 
to  consider  both  cost  and  benefit  in  evaluating  design  alternatives. 

This  project  is  unique  in  that  it  has  attempted  to  merge  the  two  concerns 
into  a  single  integrated  process.  The  model  begins  by  identifying  the  goals  of 
instruction  and  the  instructional  methods  through  which  those  goals  can  be  met. 
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It  then  assists  the  developer  in  specifying  alternative  system  designs  for 
implementing  the  instructional  methods.  In  this  way  the  model  should  produce  a 
solution  that  satisfies  both  training  and  cost  objectives. 

The  dual  concerns  of  the  model  have  led  us  to  consider  several  different 
kinds  of  issues  during  the  model  development  process.  The  remaining  issues 
concern  the  nature  of  the  model  from  the  viewpoint  of  its  simplicity  and  accuracy, 
the  ability  of  the  model  to  be  used  as  a  decision  support  system,  and  the  need  for 
future  research  on  skill  acquisition  and  transfer  of  skill  training  to  task 
performance. 


Modeling  Issues 

A  model  must  strike  a  balance  between  simplicity  and  validity  in  its 
representation  of  skill  acquisition  and  performance.  On  the  one  hand,  the  model 
must  be  able  to  capture  the  critical  interactions  between  skills  and  tasks  that 
makes  skill  training  effective  under  the  proper  conditions.  On  the  other  hand,  an 
overly  complex  model  may  increase  the  burden  for  parameter  estimation,  and  may 
reflect  more  details  about  the  skill  acquisition  process  than  can  be  justified  from 
the  research  literature.  We  began  with  very  simple  formulations  but  ran  into 
numerous  difficulties.  For  instance,  by  ignoring  the  fact  that  skills  are  inevitably 
trained  to  some  degree  during  task  training,  a  very  simplistic  model  overestimated 
the  impact  of  skill-based  training  across  multiple  tasks.  As  another  example,  in  a 
first  approach  to  skill-based  strategy  prioritization,  «ve  were  unable  to  guarantee 
that  strategies  originally  eliminated  as  inefficient  might  not  become  efficient  if 
other  skill-based  strategies  were  invoked.  Inevitably,  such  problems  led  to  more 
complex  resource  allocation  modeling. 

We  have  not  detailed  the  skill  impact  evaluation  framework  introduced  in 
the  training  allocation  activity.  The  precise  nature  of  this  framework  will  depend 
primarily  on  the  status  of  skill-to-task  transfer  research,  but  must  also  take  into 
account  the  time  and  resources  available  for  additional  research,  and  a  careful 
evaluation  of  alternate  approaches. 

Whatever  skill-based  training  evaluation  framework  is  employed,  it  must 
address  the  three  ways  in  which  skill-based  training  can  impact  total  training 
costs,  as  discussed  in  the  rationale  for  skill-based  training:  (a)  time  compression, 
(b)  generalization,  (c)  reduction  of  attention  requirements.  In  addition,  the 
framework  must  support  reasonable  data  gathering  and  estimation  procedures. 

As  part  of  the  current  research  we  roughly  outlined  several  evaluation 
frameworks.  We  now  describe  two  of  these  in  more  detail  to  indicate  the  nature 
and  breadth  of  frameworks  for  evaluating  the  worth  of  skill-based  training 
approaches.  Because  of  the  preliminary  nature  of  these  frameworks,  they  are  not 
presented  in  detail. 
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Simple  Linear  Weighting  Scheme 


In  this  technique  an  estimate  is  made  of  the  importance  of  constituent  skills 
to  learning  of,  and  performance  on,  a  particular  task.  If  skill  training  increases 
the  entry  level  of  a  particular  skill  within  task  performance,  that  increase  is 
multiplied  by  the  importance  of  the  skill  to  the  task  to  get  a  measure  of  value  for 
that  skill  increase  for  that  task.  In  the  same  way  the  value  of  the  skill  increase  to 
each  other  task  in  which  the  skill  is  used  is  estimated.  A  final  measure  of  value 
for  the  skill  increase  is  obtained  by  weighting  the  value  to  each  task  by  the 
importance  of  that  task,  perhaps  in  terms  of  training  dollars.  This  simple 
weighting  factor  approach  is  then  used  to  compare  and  select  among  STS/device 
candidates. 

The  advantages  of  this  approach  include  simplicity,  apparent 
understandability  and  face  validity,  easy  implementation,  and  fast  computation. 

A  "typical"  weighting  scheme  approach,  it  may  win  acceptance  based  on 
familiarity.  The  disadvantages  rest  in  its  largely  heuristic  nature,  which  makes  it 
difficult  to  explicate  exactly  how  a  skill  actually  impacts  task  performance;  also, 
there  is  reason  to  believe  that  this  scheme  will  tend  to  overstate  multiple-task 
impact  by  ignoring  the  skill  training  that  occurs  during  task  training. 

Detailed  Learning  Curve  Construction  Approach 

We  investigated  a  skill  impact  evaluation  framework  based  on  the  detailed 
decomposition  of  task  learning  curves  into  constituent  skill  learning  curves.  We 
acsumed  that,  during  training,  each  increment  of  student  time  is  partitioned 
among  the  constituent  skills  and  the  residual  task-specific  component,  which  is 
also  accorded  an  independent  learning  curve.  The  fraction  of  time  apportioned  to 
a  particular  component  is  made  to  depend  both  on  the  importance  of  that 
component  and  on  the  "deficit  performance"  of  that  skill,  i.e.,  the  deviation 
between  current  and  desired  performance.  The  learning  accomplished  on  a 
particular  task  component  is  then  a  function  of  the  total  time  devoted  to  that 
component  from  the  start  of  training;  the  cumulative  time  devoted  to  training  one 
component  will  be  less  than  the  total  time  involved  in  training  the  task,  since  time 
is  split  among  the  constituent  components.  If  skill  training  increases  the  entry 
performance  on  a  particular  skill  within  a  task,  then  the  learning  rate  on  that 
component  will  be  reduced,  while  that  on  the  other  component  skills  will  increase. 
This  occurs  naturally  as  student  attention  is  focussed  most  on  the  least  well 
mastered  task  components.  In  consequence,  total  task  performance  will  increase 
more  quickly.  Upon  reaching  task  criterion  performance,  the  level  of  skill 
performance  will  be  somewhat  higher  than  would  be  the  case  had  no  prior  skill 
training  been  undertaken.  This  increase  in  skill  performance  will  be  transferred 
to  the  next  task  in  the  training  sequence,  thereby  causing  a  subsequent 
acceleration  in  the  training  of  that  task.  In  this  way  the  multiple  task 
generalization  is  handled,  without  overestimating  its  impact.  In  addition  to 
dealing  with  time  compression  and  generalization,  the  framework  can  be  extended 
to  account  for  attentional  overload. 
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The  primary  advantage  of  the  learning-curve  based  approach  is  its  explicit 
mathematical  description  of  the  linkage  between  skill  and  task  performance.  This 
makes  it  easy  for  the  mathematically-trained  reader  to  quantitatively  understand 
the  three  ways  in  which  skill-based  training  can  impact  total  training  costs. 
Disadvantages  include  inaccessibility  to  many  readers,  increased  computational 
load  (though  this  may  not  prove  important),  and  the  larger  number  of  parameters 
to  be  estimated.  The  entire  approach  is,  apparently,  a  novel  extension  of 
traditional  learning  curve  formulations;  as  such,  its  reception  and  endorsement  by 
other  researchers  is  not  assured. 


User  Issues 


To  be  successful  as  the  basis  of  a  decision  support  system,  a  model  must 
provide  information  that  is  both  accurate  and  pertinent  to  the  training  design 
problem  at  hand.  Furthermore  the  effort  required  to  obtain  the  information  must 
be  commensurate  with  the  value  of  the  results.  No  system  will  solve  training 
design  problems  without  some  effort  from  the  user.  However,  systems  that 
require  substantial  effort  must  provide  results  of  similar  value. 

Relevance 


Relevance  is  a  primary  concern  in  the  evaluation  of  any  decision  aid. 
Ensuring  relevance  requires  contact  with  potential  users  of  the  aid.  At  this  stage 
in  the  development  process,  is  too  early  to  determine  the  relevance  of  the  decision 
model  presented  in  this  report.  However,  it  is  critical  that  before  the  model  is 
developed  further,  potential  users  are  allowed  to  evaluate  the  conceptual  model. 

Data  Requirements 

The  skill-based  training  system  design  model  requires  considerable  data. 
Many  of  the  data  requirements  can  be  met  by  standard  task  analytic  methods,  but 
other  requirements  are  outside  of  the  scope  of  traditional  task  analysis.  This 
discussion  focuses  on  three  types  of  data  required  by  the  model:  (a)  detailed 
process  descriptions,  (b)  quantitative  estimates,  and  (c)  cost  estimates. 

The  primary  role  of  the  SME  in  the  model  is  to  provide  detailed  descriptions 
of  how  general  abilities  are  applied  to  specific  tasks.  In  Air  Traffic  Control,  for 
example,  the  SME  would  be  required  to  determine  whether  the  controller  senses 
potential  aircraft  conflicts  using  an  analog  visual  model  of  the  airspace,  or 
whether  the  controller  performs  a  more  analytical  analysis  of  aircraft  information. 
This  information  is  obviously  critical  for  the  design  of  training  for  this  activity.  It 
is  equally  obvious  that  the  SME  is  the  only  possible  source  of  the  required 
information.  The  procedure  to  obtain  information  from  the  SME  in  the  model 
extends  traditional  task  analytic  methods  that  describe  tasks,  subtasks,  and  task 
elements.  By  carefully  eliciting  the  required  information  so  that  the  effects  of 
SME  biases  are  minimized,  it  should  be  possible  to  obtain  the  detailed 
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descriptions  of  skill  performance  required  by  the  model  with  reasonable 
confidence. 

Quantitative  estimates  are  inherently  more  difficult  to  obtain  than  verbal 
data  or  ordinal  comparisons,  because  a  single  quantitative  estimate  has  the 
information  of  several  ordinal  comparisons.  The  process  of  obtaining  quantitative 
estimates  has  been  refined  over  60  years  of  scaling  research,  and  it  should  be 
possible  to  develop  scaling  procedures  for  most  quantitative  variables  that  are 
both  reasonably  accurate  and  easily  applied.  These  methods  include  procedures 
that  allow  the  researcher  to  infer  the  quantitative  information  that  is  present  in 
rank  orders  of  sets  of  objects,  procedures  that  compare  objects  to  pre-scaled  verbal 
anchors,  or  procedures  in  which  the  judge  compares  specially  chosen  combinations 
of  objects.  Thus,  although  obtaining  estimates  of  quantitative  parameters  is 
difficult  because  of  the  information  those  estimates  contain,  there  are  a  variety  of 
procedures  available  to  obtain  these  estimates  in  a  relatively  painless  fashion. 

Estimates  of  cost  may  be  the  most  difficult  data  requirements  of  the  model. 
In  fact,  it  is  probably  the  difficulty  of  obtaining  accurate  cost  estimates  that  leads 
most  procedures  for  training-system  design  to  ignore  cost  until  most  (or  all) 
instructional  decisions  have  been  made.  The  reasons  for  the  difficulty  in  obtaining 
cost  information  are  manifold.  Among  these  difficulties  are  problems  in 
determining  which  cost  elements  to  include  in  the  estimate,  the  near  impossibility 
in  obtaining  an  accurate  cost  accounting  for  existing  training  systems,  and  the 
difficulty  in  predicting  the  future  cost  of  technology.  One  aspect  of  the  model  that 
makes  the  cost-estimation  problem  easier  is  that  the  model  requires  that  cost  be 
measured  only  on  a  ratio  scale.  That  is,  the  costs  may  reflect  relative  costs,  as 
compared  to  an  arbitrary  standard.  Since  only  relative  costs  are  required,  we 
recommend  that  a  relative  scaling  procedure  be  used  to  obtain  costs.  Knerr, 
Morrison,  Mumaw,  Sticha,  Blacksten,  Harris,  and  Lahey  (1987)  applied  such  a 
procedure  to  estimate  the  cost  of  developing  instructional  support  features.  This 
procedure  was  adapted  by  Braby,  Charles,  Sylla,  Ramesh,  Willis,  and  Hunter 
(1988)  for  design  of  training-device  instructor/operator  station. 

Research  Issues 


The  skill-based  training  system  design  model  is  based  on  a  combination  of 
problem  analysis  and  empirical  research.  Neither  of  these  two  methods  is 
sufficient  without  support  from  the  other.  There  are  two  primary  areas  where  the 
need  for  further  empirical  research  is  obvious:  (a)  transfer  of  training  between 
tasks  and  skills,  and  (b)  effect  of  limited  attention  and  overload  on  learning. 
Further  research  in  these  areas  is  critical  to  further  our  understanding  of  the 
capabilities  of  skill-based  training. 

Transfer  of  Training 

Skill  training  is  based  on  the  assumption  that  it  is  possible  to  get 
significant  transfer  of  training  between  activities  trained  under  substantially 
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different  conditions.  Determining  the  extent  to  which  such  transfer  is  possible 
and  the  conditions  under  which  it  will  occur  touches  on  an  issue  that  has  been 
alive  in  the  psychological  literature  for  nearly  a  century.  Recent  theories  of 
transfer  of  training,  such  as  Kieras  and  Bovair  (1986)  postulate  that  transfer  of 
training  occurs  at  the  microscopic  level  of  the  individual  production  rule.  Our 
analysis  has  been  consistent  with  this  theory  in  the  sense  that  we  conduct  a  skill 
analysis  down  to  the  level  of  the  specific  skill,  which  specifies  both  the  general 
ability  involved  and  the  domain-specific  knowledge  on  which  the  ability  operates. 

However,  it  is  not  feasible  to  continue  the  skill  analysis  to  the  level  of  the 
single  production  rule;  the  time  required  for  such  an  analysis  would  be  well 
beyond  the  scope  of  any  task-analytic  effort.  Our  approach  to  transfer  of  training 
uses  a  very  general  characterization  of  transfer  as  the  asymptote  of  a  function,  or 
equivalently  as  a  slope  of  a  linear  transfer  function.  It  is  difficult  to  estimate  the 
accuracy  of  our  simple  assumption.  One  goal  of  the  skill  characterization  is  to 
provide  a  heuristic  for  estimating  transfer  of  training.  It  is  not  clear  that  the  skill 
characterization  has  sufficient  depth  at  this  point  to  aid  transfer  estimation. 

The  problem  of  transfer  of  training  is  especially  critical  when  one  must 
allocate  training  resources  among  skill-training  and  task-training  options.  In  this 
case  transfer  may  occur  between  tasks,  between  skills,  or  between  tasks  and 
skills.  Finding  the  optimal  allocation  of  training  resources  requires  us  to 
understand  how  these  sources  of  transfer  balance  out  for  any  allocation  option.  As 
the  previous  discussion  indicated,  accounting  for  transfer  of  training  is  a  potential 
source  of  substantial  model  complexity.  A  thorough  empirical  research  program  is 
required  to  determine  the  appropriate  rules  to  characterize  transfer  of  training. 

Attention  and  Overload 

We  have  assumed  that  the  attention  required  to  perform  or  learn  a  task 
decreases  in  proportion  to  the  level  of  performance.  This  assumption  reflects 
general  empirical  results  and  theoretical  formulations,  but  the  specific  nature  of 
the  assumption  is  completely  ad  hoc.  There  is  reason  to  believe  that  attention 
requirements  decrease  at  a  slower  rate  than  performance  improves,  contrary  to 
our  assumptions.  There  is  also  reason  to  postulate  that  controlled  and  automatic 
processes  are  separate  processes,  which  may  develop  independently,  or  as  a  result 
of  a  specific  knowledge  compilation  process.  Either  of  these  changes  would 
increase  the  complexity  of  the  model  significantly.  This  area  of  skill  acquisition  is 
one  where  current  research  trends  are  likely  to  produce  useful  results.  It  is 
important  that  these  research  programs  continue,  and  that  training-system 
designers  are  informed  of  these  results. 

Conclusions  and  Recommendations 


We  have  developed  an  integrated  framework  for  making  decisions  regarding 
the  design  of  skill-based  training  systems.  The  decision-making  procedure 
addresses  the  identification  of  skills  and  training  strategies,  the  design  of  training 
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devices,  and  the  allocation  of  training  resources  to  skill-based  and  task-based 
training  alternatives.  Underlying  the  decision  model  are  a  skill  characterization 
and  cost-effectiveness  model  that  represent  many  of  the  factors  that  are  important 
in  evaluating  skill  training  options. 

The  need  for  a  long-term  research  effort  is  obvious  from  the  development  at 
this  point.  There  are  several  topics  for  further  research  which  would  enhance  our 
abilities  to  predict  the  cost  and  effectiveness  of  skill  training,  including  research 
on  transfer  of  training  from  skill  training  to  task  performance,  and  research  on 
the  effects  of  attention  and  overload. 

Clearly,  the  model  is  in  its  infancy.  Although  grounded  in  a  strong  body  of 
research,  we  know  that  the  accuracy  with  which  we  may  extrapolate  from  theory 
and  prior  experience  is  limited.  The  model  makes  several  assumptions  that  will 
need  to  be  tested  and  revised.  Its  usability  in  the  field  also  must  be  studied  and 
enhanced.  We  believe  that  the  best  way  to  proceed  is  to  apply  the  model  to 
several  task  domains  chosen  not  for  high  probability  of  success,  but  rather  on  the 
basie  of  job  complexity  and  training  need.  Researchers  need  to  work  with  training 
developers  to  find  ways  to  tailor  the  model  to  their  needs.  Through  this  iterative 
process  the  Army  can  develop  the  tools  to  develop  more  effective  and  efficient 
training. 
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APPENDIX 

DEFINITIONS  OF  GENERAL  ABILITIES 


These  definitions  of  the  general  abilities  are  drawn  primarily  from 
Fleishman  &  Quaintance  (1984)  and  Miller  (1971).  Most  of  the  definitions  are 
quoted  directly  from  the  cited  sources;  any  adaptations  are  r.^-ed.  Definitions 
without  attribution  are  combined  from  several  sources. 

Psychomotor  Abilities 

Control  precision.  The  ability  to  make  fine,  highly  controlled 
muscular  movements  required  to  adjust  the  position  of  a  control 
mechanism.  Examples  of  control  mechanisms  are  joy  sticks,  levels 
[sic.],  pedals,  and  rudders  .  .  .  This  ability  is  most  critical  where 
adjustments  must  be  rapid,  but  precise  (Fleishman  &  Quaintance, 

1984,  p.  164). 

Dexterity.  The  ability  to  make  skillful,  well  directed  arm-hand 
movements  in  manipulating  fairly  large  objects  under  speeded 
conditions  (Fleishman  &  Quaintance,  1984,  p.  165). 

Multilimb  coordination.  The  ability  to  coordinate  the  movement  of  a 
number  of  limbs  simultaneously  (Fleishman  &  Quaintance,  1984,  p. 
164). 

Rate  control.  [This  ability]  involves  the  timing  of  continuous 
anticipatory  motor  adjustments  relative  to  changes  in  speed  and/or 
direction  of  a  continuously  moving  target  or  object  (Fleishman  & 
Quaintance,  1984,  p.  164). 

Reaction  speed.  This  ability  represents  the  speed  with  which  the 
individual  can  provide  a  single  motor  response  to  a  single  stimulus 
when  it  appears  (Fleishman  &  Quaintance,  1984,  p.  164). 

Aiming.  The  ability  to  make  highly  accurate,  restricted  hand 
movements  requiring  precise  eye-hand  coordination  (Fleishman  & 
Quaintance,  1984,  p.  165). 

Response  orientation.  This  is  the  ability  to  select  and  initiate  the 
appropriate  response  relative  to  a  given  stimulus  in  the  situation 
where  two  or  more  stimuli  are  possible  and  where  the  appropriate 
response  is  selected  from  two  or  more  alternatives.  The  ability  is 
concerned  with  the  speed  with  which  the  appropriate  response  can  be 
initiated  and  does  not  extend  to  the  speed  with  which  the  response  is 
carried  out  (Fleishman  &  Quaintance,  1984,  pp.  324-5). 
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Perceptual  Abilities 


Detection.  [The  ability  to  sense]  the  presence  or  absence  of  a  cue  or 
condition  requiring  that  some  form  of  action  should  be  taken  by  the 
system  .  .  .  Detecting  results  in  sensing  a  stimulus  to  which  attention 
will  be  paid  (Miller,  1971,  cited  in  Fleishman  &  Quaintance,  1984,  p. 
442). 

Selective  attention.  This  is  the  ability  to  perform  a  task  in  the 
presence  of  distracting  stimulation  or  under  monotonous  conditions 
without  significant  loss  in  efficiency  .  .  .  Under  conditions  of 
distracting  stimulation,  the  ability  involves  concentration  on  the  task 
being  performed  and  filtering  out  of  the  distracting  stimulation. 
When  the  task  is  performed  under  monotonous  conditions  only 
concentration  on  the  task  being  performed  is  involved  (Fleishman  & 
Quaintance,  1984,  p.  323). 

Spatial  reasoning.  The  ability  to  make  inferences  about  the  absolute 
or  relative  positions  and  velocities  of  objects  in  space. 

Pattern  recognition.  [The  ability  to  characterize]  a  message  by  type 
or  source  ...  [to  identify]  an  object  or  entity  and  apply  some  label  to 
it  (Miller,  1971,  cited  in  Fleishman  &  Quaintance,  1984,  p.  444). 

Visualization.  The  ability  to  imagine  how  something  will  look  when 
it  is  moved  around  or  when  its  parts  are  moved  or  rearranged.  It 
requires  the  forming  of  mental  images  of  how  patterns  or  objects 
would  look  after  certain  changes,  such  as  unfolding  or  rotation.  One 
has  to  predict  how  an  object,  set  of  objects,  or  pattern  will  appear 
after  the  changes  are  carried  out  (Fleishman  &  Quaintance,  1984,  p. 
462). 

Discrimination.  [The  ability  to]  classify  or  differentiate  an  entity  in 
terms  of  gross-level  grouping  or  set  membership,  frequently  on  the 
basis  of  only  a  limited  number  of  attributes  (Ammerman,  et  al., 
1987). 

Cognitive  Abilities 

Verbal  comprehension.  The  ability  to  understand  language.  It  is 
concerned  with  the  understanding  of  individual  words  as  well  as 
words  as  they  appear  in  context,  i.e.,  in  sentences,  grammatical 
patterns  and  idiomatic  phrases  (Fleishman  &  Quaintance,  1984,  p. 
322). 

Quantitative  reasoning.  The  ability  to  reason  abstractly  using 
quantitative  concepts  and  symbols.  It  encompasses  reasoning 
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through  mathematical  problems  in  order  to  determine  appropriate 
operations  which  can  be  performed  to  solve  them.  It  also  includes  the 
understanding  or  structuring  of  mathematical  problems.  The  actual 
manipulation  of  numbers  is  not  included  in  this  ability  (Fleishman  & 
Quaintance,  1984,  p.  322). 

Meta-cognition.  The  ability  to  monitor  internal  learning  and  cognitive 
processes. 

Inductive  reasoning.  The  ability  to  find  the  most  appropriate  general 
concepts  or  rules  which  fit  sets  of  data  or  which  explain  how  a  given 
series  of  individual  items  are  related  to  each  other.  It  involves  the 
ability  to  synthesize  disparate  facts;  to  proceed  logically  from 
individual  cases  to  general  principles  (Fleishman  &  Quaintance,  1984, 
pp.  322-3). 

Analogical  reasoning.  The  ability  to  formulate  a  solution  to  a 
problem  by  comparing  the  object  of  the  problem  with  another  object 
(the  analog)  that  has  similar  properties,  and  performing  actions  on 
the  object  of  the  problem  that  correspond  to  actions  that  would  be 
performed  on  the  analog. 

Prioritizing.  [The  ability  to  order]  events  in  sequence;  establishing 
priorities  (Ammerman,  et  al.,  1987). 

Information  recall.  The  ability  to  remember  information,  such  as 
words,  numbers,  pictures,  and  procedures.  Pieces  of  information  can 
be  remembered  by  themselves  or  with  other  pieces  of  information 
(Fleishman  &  Quaintance,  1984,  p.461). 

Backward  chaining.  The  ability  to  formulate  a  solution  to  a  problem 
by  analyzing  goals  into  subgoals,  and  selecting  actions  that  meet  the 
subgoals. 

Abstraction.  The  ability  to  determine  the  formal  or  logical 
characteristics  of  a  situation  or  object. 

Computation.  The  ability  to  manipulate  numbers  in  numerical 
operations;  for  example,  add,  subtract,  multiply,  divide,  integrate, 
differentiate,  etc.  The  ability  involves  both  the  speed  and  accuracy  of 
computation  (Fleishman  &  Quaintance,  1984,  p.  322). 

Time  sharing.  The  ability  to  shift  back  and  forth  between  two  or 
more  sources  of  information  (Fleishman  &  Quaintance,  1984,  p.  463). 
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Deductive  reasoning.  The  ability  to  apply  general  concepts  or  rules  to 
specific  cases  or  to  proceed  from  stated  premised  to  their  logical 
conclusions  (Fleishman  &  Quaintance,  1984,  p.  322). 

Planning.  [The  ability  to  predict]  what  future  sets  of  conditions  will 
occur  and  what  responses  to  make  to  them  and  in  what  order  (Miller, 
1971,  cited  in  Fleishman  &  Quaintance,  1984,  p.452). 

Knowledge  compilation.  The  ability  to  convert  declarative  knowledge 
about  how  to  perform  an  activity  into  a  procedural  representation 
that  produces  an  increase  in  performance  speed  (adapted  from 
Anderson,  1983). 

Forward  chaining.  The  ability  to  formulate  the  solution  to  a  problem 
by  selecting  actions  that  reduce  the  scope  of  the  problem,  until  the 
goal  is  attained. 

Categorization.  The  ability  to  classify  data,  information,  or 
intelligence  according  to  its  source,  format,  purpose,  or  content  in 
order  to  organize  messages  into  meaningful  groups,  or  in  order  to 
selectively  retrieve  them  for  decision  making  and  control  (Miller, 
1971,  cited  in  Fleishman  &  Quaintance,  1984,  p.  446). 
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